Skip to content

Drop the HF initial_occupancies seed; ship a curated H2O counts file - #13

Merged
hfwen0502 merged 1 commit into
mainfrom
remove-hf-occupancy-seed
Sep 1, 2026
Merged

Drop the HF initial_occupancies seed; ship a curated H2O counts file#13
hfwen0502 merged 1 commit into
mainfrom
remove-hf-occupancy-seed

Conversation

@hfwen0502

Copy link
Copy Markdown
Collaborator

The problem

run_sqd_sbd.py passed initial_occupancies unconditionally, described in a comment as
"cheap insurance … harmless if the bit_array is already valid". It is not harmless.
qiskit_addon_sqd.fermion._prepare_ci_strings branches on it:

if current_occupancies is None:
    # KEEP only bitstrings that already have the target Hamming weights
    bitstrings, probs = postselect_by_hamming_right_and_left(...)
else:
    # take the FULL noisy set and bit-flip every string toward the target
    bitstrings, probs = recover_configurations(...)

So supplying it replaced postselection with configuration recovery in every run,
--counts runs included. At iteration 1 the only occupancies available are
Hartree–Fock (every virtual exactly 0.0), so the subspace was largely synthesised
by bit-flips against a mean-field prior rather than built from sampled
configurations.

Published SQD postselects at iteration 1 and uses solver-derived occupancies from
iteration 2 onward. qiskit-addon-dice-solver's test_n2_sbd.py already does
exactly that — it passes no initial_occupancies.

Changes

  • run_sqd_sbd.py: drop the HF occupancy block and the initial_occupancies=
    argument.

  • run_sqd_sbd.py: the --samples fallback now uses
    generate_counts_bipartite_hamming() instead of generate_bit_array_uniform(),
    so generated bitstrings carry the target Hamming weights. Uniform strings are
    postselected away except near half filling — survival is
    C(norb,ne)^2 / 4^norb:

    system norb e-/spin valid per 10k uniform
    test_n2_sbd 16 5 44
    atom_1752 45 23 137
    H2O (data/h2o) 24 5 0.1
    N2 (data/n2) 60 7 0

    Measured: 0 of 3000 uniform vs 3000 of 3000 with the new generator (H2O).
    Three stale "uniform random" descriptions (docstring, --samples help, runtime
    print) are corrected.

  • New python/examples/count_dict_h2o.json (15 KB): 275 bitstrings in
    [beta | alpha] layout, built from the vendored h2o-1em3-alpha.txt list with
    each alpha half-determinant reused for both spin sectors — the same
    tensor-product subspace run_sbd_diag.py builds from --adetfile. Counts are
    all 1: a curated determinant list, not a shot histogram.

  • Notebook: loads that file instead of generating random bitstrings, and drops
    initial_occupancies. It previously reached ~-76.2 only because the HF seed
    pulled uniform-random strings toward HF; with the seed removed and random
    samples it would land near -31 Ha. Also relabels the callback's
    subspace dim -> carryover dim, which reports the 27x27 carryover set rather
    than the 75,625 determinants actually diagonalized.

Validation

The notebook was executed locally with the same tooling CI uses
(pytest --nbmake, cpu backend, stock qiskit-addon-sqd 0.13.1,
pyscf 2.14.0, python 3.13.5):

1 passed in 8.31s

cell  6| Loaded 275 bitstrings (48 bits / shot) from count_dict_h2o.json
cell  6| (alpha, beta) Hamming weights present: {(5, 5)}  -> target (5, 5)
cell  6| Tensor-product subspace: 275 x 275 = 75_625 determinants
cell 10|   iter 1, batch 0:  E = -76.2359466308 Ha   carryover dim = 729
cell 10| Final SQD energy:  -76.2359466308 Ha

SBD's own output reads Energy = -76.23594663084391, matching the
-76.23594663 that vendor/sbd-upstream/data/h2o/README.md publishes for the
1.0e-3 / 275-bitstring / 7.56e4-determinant row.

Runtime on an M1 Max: 7.8 s total (0.5 s kernel, 1.7 s imports, 5.0 s for the two
SQD iterations; Davidson is 2.34 s + 2.31 s).

All 275 bitstrings survive postselection, so no occupancy seed is needed. Also
verified end-to-end on GB200 with --device gpu: -76.2359466308.

Notes for review

  • Counts are all 1 rather than amplitude-like weights: the FCI coefficients aren't
    in the repo, and inventing plausible-looking weights would misrepresent the file.
  • Because samples_per_batch=300 >= 275, subsample() returns the full set, so
    both iterations diagonalize the same subspace and return the same energy. That
    makes the notebook deterministic and exactly reproducible against the upstream
    number, but it does mean the loop does not visibly improve across iterations.

initial_occupancies was passed unconditionally, which is not the inert
"insurance" its comment claimed. qiskit_addon_sqd._prepare_ci_strings
branches on it:

  None      -> postselect_by_hamming_right_and_left(): keep only the
               bitstrings that already have the target Hamming weights.
  provided  -> recover_configurations(): take the full noisy set and
               bit-flip every string toward the target.

So supplying it replaced postselection with configuration recovery in
every run, including --counts runs. At iteration 1 the only occupancies
available are Hartree-Fock (all virtuals exactly 0.0), so the subspace
was largely synthesised by bit-flips against a mean-field prior instead
of built from sampled configurations. Published SQD postselects at
iteration 1 and uses solver-derived occupancies from iteration 2 on,
which is what qiskit-addon-dice-solver's test_n2_sbd.py already does.

Changes:
- run_sqd_sbd.py: drop the HF occupancy block and the
  initial_occupancies= argument.
- run_sqd_sbd.py: the --samples fallback now uses
  generate_counts_bipartite_hamming() instead of
  generate_bit_array_uniform(), so generated bitstrings carry the target
  Hamming weights. Uniform strings are postselected away except near half
  filling: the survival rate is C(norb,ne)^2 / 4^norb, i.e. 0 of 3000 for
  H2O (5 of 24) versus 3000 of 3000 with this generator.
- Add python/examples/count_dict_h2o.json: 275 bitstrings in
  [beta | alpha] layout, from the vendored h2o-1em3-alpha.txt list with
  each alpha half-determinant reused for both spin sectors. Counts are
  all 1 -- a curated determinant list, not a shot histogram.
- Notebook: load that file instead of generating random bitstrings, and
  drop initial_occupancies. Previously it reached ~-76.2 only because the
  HF seed pulled uniform-random strings toward HF.

Validated (GB200, --device gpu, notebook's settings: samples_per_batch
300, 1 batch, 2 iterations): -76.2359466308 Ha, matching the -76.23594663
that vendor/sbd-upstream/data/h2o/README.md reports for the 1.0e-3 /
275-bitstring / 7.56e4-determinant subspace. All 275 bitstrings survive
postselection, so no occupancy seed is required.
@hfwen0502
hfwen0502 force-pushed the remove-hf-occupancy-seed branch from 1b44081 to 49eb23e Compare September 1, 2026 19:01
@hfwen0502
hfwen0502 merged commit 29be7e5 into main Sep 1, 2026
6 checks passed
@hfwen0502
hfwen0502 deleted the remove-hf-occupancy-seed branch September 1, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant