Skip to content

pass_configlink: CONFIGURES edges are nondeterministic when candidates exceed the collector caps #1468

Description

@Studnicky

Indexing the same corpus multi-threaded produces a different CONFIGURES edge set on every run, and a different set from the single-threaded reference. All observed divergence is CONFIGURES; every other edge type is byte-stable.

This is the same defect class as #998 (SEMANTICALLY_RELATED churn), in a pass that did not get the canonical-ordering treatment.

Reproduction

Corpus: 4191-file TypeScript tree, --mode moderate, each run into a fresh CBM_CACHE_DIR. Fingerprint is the sorted (source_qn, type, target_qn) triple set read straight from the project db, compared with LC_ALL=C.

run edges vs single-threaded
ST (reference) 70809
MT #1 70812 4 missing, 7 extra
MT #2 70864 6 missing, 61 extra
MT #3 70807 5 missing, 3 extra

No two multi-threaded runs agree. Across the three runs, 86 differing edges — 100% of them CONFIGURES, out of 414 CONFIGURES edges total.

Cause

strategy_key_symbols fills two fixed-capacity arrays:

config_entry_t config_entries[CBM_SZ_4K];
int config_count = collect_config_entries(vars, var_count, config_entries, CBM_SZ_4K);

code_entry_t code_entries[CBM_SZ_8K];
int code_count = collect_code_entries(gb, code_entries, CBM_SZ_8K);

Both collectors stop at the cap:

for (int li = 0; labels[li] && n < max_out; li++)      /* Function, Variable, Class, Struct */
    for (int i = 0; i < count && n < max_out; i++)     /* silently stops at 8192 */

They walk cbm_gbuf_find_by_label in gbuf insertion order, which under parallel extraction is worker-merge order and varies run to run. On this corpus there are 12940 candidates (Function 7121, Variable 5524, Class 295) against the 8192 cap, so 4748 are dropped — and which ones depends on thread scheduling. Functions consume the first ~7121 slots, leaving roughly 1071 for Variables, so the cut lands mid-group where the ordering is unstable.

Confirming the mechanism

A corpus with 2871 candidates — below the cap — is byte-identical across the single-threaded reference and three multi-threaded runs. The divergence appears only once the cap binds.

Suggested fix

The same recipe #998 called for: order candidates by a pure content key before applying the cap, so the surviving set is a function of the inputs rather than of worker scheduling. Tie-breaks need to be content-only — node ids are handed out in merge order, so an id tie-break would reintroduce the same dependence.

Worth surfacing the truncation too; at present the caps drop candidates with no signal at all.

Notes

  • The count delta understates the divergence: losses and gains partly cancel, so a run can differ in dozens of edges while the total moves by three.
  • tests/repro/repro_parallel_determinism.c covers the general parallel-determinism property but is pinned to an absolute corpus path and skips when absent, so this pass is not currently guarded in CI.

Environment: v0.9.0 (9b9b6ff), darwin-arm64, built from source.

Related: #998, #1085, #921, #923.

Metadata

Metadata

Assignees

No one assigned

    Labels

    parsing/qualityGraph extraction bugs, false positives, missing edges

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions