Indexing the same corpus multi-threaded produces a different CONFIGURES edge set on every run, and a different set from the single-threaded reference. All observed divergence is CONFIGURES; every other edge type is byte-stable.
This is the same defect class as #998 (SEMANTICALLY_RELATED churn), in a pass that did not get the canonical-ordering treatment.
Reproduction
Corpus: 4191-file TypeScript tree, --mode moderate, each run into a fresh CBM_CACHE_DIR. Fingerprint is the sorted (source_qn, type, target_qn) triple set read straight from the project db, compared with LC_ALL=C.
| run |
edges |
vs single-threaded |
| ST (reference) |
70809 |
— |
| MT #1 |
70812 |
4 missing, 7 extra |
| MT #2 |
70864 |
6 missing, 61 extra |
| MT #3 |
70807 |
5 missing, 3 extra |
No two multi-threaded runs agree. Across the three runs, 86 differing edges — 100% of them CONFIGURES, out of 414 CONFIGURES edges total.
Cause
strategy_key_symbols fills two fixed-capacity arrays:
config_entry_t config_entries[CBM_SZ_4K];
int config_count = collect_config_entries(vars, var_count, config_entries, CBM_SZ_4K);
code_entry_t code_entries[CBM_SZ_8K];
int code_count = collect_code_entries(gb, code_entries, CBM_SZ_8K);
Both collectors stop at the cap:
for (int li = 0; labels[li] && n < max_out; li++) /* Function, Variable, Class, Struct */
for (int i = 0; i < count && n < max_out; i++) /* silently stops at 8192 */
They walk cbm_gbuf_find_by_label in gbuf insertion order, which under parallel extraction is worker-merge order and varies run to run. On this corpus there are 12940 candidates (Function 7121, Variable 5524, Class 295) against the 8192 cap, so 4748 are dropped — and which ones depends on thread scheduling. Functions consume the first ~7121 slots, leaving roughly 1071 for Variables, so the cut lands mid-group where the ordering is unstable.
Confirming the mechanism
A corpus with 2871 candidates — below the cap — is byte-identical across the single-threaded reference and three multi-threaded runs. The divergence appears only once the cap binds.
Suggested fix
The same recipe #998 called for: order candidates by a pure content key before applying the cap, so the surviving set is a function of the inputs rather than of worker scheduling. Tie-breaks need to be content-only — node ids are handed out in merge order, so an id tie-break would reintroduce the same dependence.
Worth surfacing the truncation too; at present the caps drop candidates with no signal at all.
Notes
- The count delta understates the divergence: losses and gains partly cancel, so a run can differ in dozens of edges while the total moves by three.
tests/repro/repro_parallel_determinism.c covers the general parallel-determinism property but is pinned to an absolute corpus path and skips when absent, so this pass is not currently guarded in CI.
Environment: v0.9.0 (9b9b6ff), darwin-arm64, built from source.
Related: #998, #1085, #921, #923.
Indexing the same corpus multi-threaded produces a different
CONFIGURESedge set on every run, and a different set from the single-threaded reference. All observed divergence isCONFIGURES; every other edge type is byte-stable.This is the same defect class as #998 (
SEMANTICALLY_RELATEDchurn), in a pass that did not get the canonical-ordering treatment.Reproduction
Corpus: 4191-file TypeScript tree,
--mode moderate, each run into a freshCBM_CACHE_DIR. Fingerprint is the sorted(source_qn, type, target_qn)triple set read straight from the project db, compared withLC_ALL=C.No two multi-threaded runs agree. Across the three runs, 86 differing edges — 100% of them
CONFIGURES, out of 414CONFIGURESedges total.Cause
strategy_key_symbolsfills two fixed-capacity arrays:Both collectors stop at the cap:
They walk
cbm_gbuf_find_by_labelin gbuf insertion order, which under parallel extraction is worker-merge order and varies run to run. On this corpus there are 12940 candidates (Function 7121, Variable 5524, Class 295) against the 8192 cap, so 4748 are dropped — and which ones depends on thread scheduling. Functions consume the first ~7121 slots, leaving roughly 1071 for Variables, so the cut lands mid-group where the ordering is unstable.Confirming the mechanism
A corpus with 2871 candidates — below the cap — is byte-identical across the single-threaded reference and three multi-threaded runs. The divergence appears only once the cap binds.
Suggested fix
The same recipe #998 called for: order candidates by a pure content key before applying the cap, so the surviving set is a function of the inputs rather than of worker scheduling. Tie-breaks need to be content-only — node ids are handed out in merge order, so an id tie-break would reintroduce the same dependence.
Worth surfacing the truncation too; at present the caps drop candidates with no signal at all.
Notes
tests/repro/repro_parallel_determinism.ccovers the general parallel-determinism property but is pinned to an absolute corpus path and skips when absent, so this pass is not currently guarded in CI.Environment: v0.9.0 (
9b9b6ff), darwin-arm64, built from source.Related: #998, #1085, #921, #923.