Fix bit_length=64 undefined behavior, make it configurable, and lower --samples/--max_nb defaults - #12
Merged
Merged
Conversation
…able
SBD packs determinant bitstrings into std::vector<size_t> with
`bit_length` bits per word. The Python wrapper hardcoded 64 in four
places, but bitadvance() in framework/bit_manipulation.h computes
size_t d = (((size_t) 1) << bit_length) - 1;
Shifting a 64-bit size_t by 64 is undefined behavior. In practice the
shift count is masked to 0, so the mask collapses to d == 0 (checked on
aarch64 and x86-64). bitadvance() is reached from mpi_redistribution()
and mpi_sort_bitarray(), the multi-rank determinant distribution paths.
RIKEN documents the default as 20 (see --bit_length in
apps/chemistry_tpb_selected_basis_diagonalization/README.md), which is
also what run_sbd_diag.py already defaults to; only run_sqd_sbd.py was
passing 64.
Changes:
- Add SBD_DEFAULT_BIT_LENGTH = 20 in sbd_solver.py and use it for the
_create_sbd_config default.
- Thread the effective value through. _ci_strings_to_sbd_dets() and
_sbd_dets_to_ci_strings() now take bit_length, and _solve_sci_core()
builds the config before packing so determinants are packed with the
same value the C++ engine uses to interpret them. Previously a
caller-supplied bit_length reached the engine but not the packing,
which would silently corrupt the determinants.
- Add --bit_length to run_sqd_sbd.py (default 20) instead of hardcoding.
Validated on GB200 (NVHPC 25.5, cc100, 4 GPUs, CUDA-aware MPICH 5.0.1):
- H2O 1e-5 subspace, dim 303,601: -76.2437350421 at bit_length 20, both
at np=1 and 2x2, matching the pre-change value and run_sbd_diag.py's
independent file-based path (-76.2437349413).
- bit_length 20 / 48 / 63 give bit-identical energies, confirming that
packing, unpacking and the engine now agree. norb=24 at bit_length=20
spans two size_t words, so this also exercises the multi-word path.
--max_nb sets the Davidson block size, and davidson_thrust.h allocates 2 * num_block vectors of length dim on the device, so GPU memory scales as dim * (2 * max_nb + ~4) * 8 bytes. 10 matches the upstream default in chemistry/tpb/sbdiag.h and cuts that workspace by ~4x versus 50.
The notebook's sbd_config hardcoded 64, the value this PR removes as undefined behavior. Its 'max_nb': 10 already matches the new CLI default.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1.
bit_length = 64is undefined behaviorbit_lengthsets how many bits SBD packs into eachsize_tof itsstd::vector<size_t>bitstring representation. The Python wrapper hardcoded 64in four places, but
bitadvance()inframework/bit_manipulation.hcomputesShifting a 64-bit
size_tby 64 is undefined behavior. In practice the shiftcount is masked to 0, so the mask collapses to
d == 0(checked on aarch64 andx86-64).
bitadvance()is reached frommpi_redistribution()andmpi_sort_bitarray()— the multi-rank determinant distribution paths.RIKEN documents the default as 20 (see
--bit_lengthinapps/chemistry_tpb_selected_basis_diagonalization/README.md), which is alsowhat
run_sbd_diag.pyalready defaults to. Onlyrun_sqd_sbd.pypassed 64.This adds
SBD_DEFAULT_BIT_LENGTH = 20and a--bit_lengthoption. The valueis not range-checked: callers can still pass 64 explicitly if they want the old
packing.
2. Latent correctness bug: the value only reached half the pipeline
bit_lengthdecides both how Python packs determinants and how the C++ engineinterprets them, so the two must agree. Previously a caller-supplied
bit_lengthreached the engine viaTPB_SBD.bit_lengthbut notfrom_string()/makestring(), which stayed at 64 — so--bit_length 48wouldhave silently corrupted every determinant.
_ci_strings_to_sbd_dets()and_sbd_dets_to_ci_strings()now takebit_length, and_solve_sci_core()builds the config before packing so bothsides read one value (
sbd_data.bit_length, the same object handed totpb_diag).3. Defaults and example
--max_nb50 → 10, matching upstreamchemistry/tpb/sbdiag.h.davidson_thrust.hallocates2 * num_blockdevice vectors of lengthdim,so GPU workspace scales as
dim * (2 * max_nb + ~4) * 8bytes — roughly a 4xreduction.
--samples10000 → 3000.run_sqd_sbd.ipynbsbd_configupdated from 64 to 20 (itsmax_nbwasalready 10). The notebook is executed by the
nbmaketox env in CI.Validation
Threading actually verified (GB200,
_create_sbd_config+ converters calleddirectly). Note that comparing energies across
bit_lengthvalues cannot provethis on its own: a silently-ignored flag also yields identical energies. These
two checks distinguish the cases.
Config reaches the object passed to
tpb_diag:Packing honours it —
norb=24, so word count must beceil(24/bit_length):The changing word count is the observable that rules out the flag being dropped,
and each det round-trips back to its original CI string.
End-to-end energy (NVHPC 25.5,
cc100, 4 GPUs, CUDA-aware MPICH 5.0.1):H2O 1e-5 subspace, dim 303,601 → -76.2437350421 at both np=1 and 2x2,
matching the pre-change value and
run_sbd_diag.py's independent file-basedpath (
-76.2437349413).bit_length20/48/63 all agree to the last digit.norb=24at the new default spans twosize_twords, so this exercises themulti-word packing path (previously always single-word at 64).
Not verified
--max_nb 10default is upstream's value and reduces memory asdescribed, but has not been checked for convergence on a large production
system; a smaller Krylov space may need more restarts.
max_nb = 50; the new default was notseparately re-validated.