research: local-min-cut-guided HNSW deletion repair (REJECT — negative result)

[ruvnet]

Summary

Nightly research run testing whether ruvector-mincut's LocalKCut (a real, recent local minimum-cut algorithm, arXiv:2510.08297) could cheaply triage which HNSW deletions need ruvector-hnsw-repair's expensive EagerRepair vs. cheap tombstoning. Result: REJECT, with precise root-cause evidence.

• Hypothesis: LocalCutGuidedRepair (new, crates/ruvector-mincut-repair) should recover most of EagerRepair's recall at a fraction of its cost, by using LocalKCut::find_cut to decide per-node whether eager repair is warranted.
• Measured: individual find_cut calls took 24.8-158.0 seconds each at the benchmark's real construction density (m0=32) — four to six orders of magnitude over the pre-registered 25%-of-delete-time overhead budget. Three independent runs (an interrupted 1,000-deletion run, an interrupted 12-deletion run, and a completed 3-deletion run) all confirm the same order of magnitude.
• Root cause, precisely identified: two compounding factors — (1) HNSW's small-world topology defeats LocalKCut's bounded-degree locality assumption, and (2) a genuine O(m) implementation defect in LocalKCut::check_cut (crates/ruvector-mincut/src/localkcut/mod.rs:368): it calls graph.edges() (materializing every edge in the graph) and linearly scans it, instead of the O(1) get_edge(u, v) lookup the graph already provides via its edge_index map.

What's in this PR

• crates/ruvector-mincut-repair — new crate implementing LocalCutGuidedRepair: DeletionStrategy, its benchmark (4-way comparison against TombstoneOnly/BatchRepair/EagerRepair), and tests (all passing).
• crates/ruvector-hnsw-repair — repair_one made pub so it can be reused directly instead of re-implemented (small, additive, tested).
• docs/adr/ADR-340-mincut-guided-hnsw-repair.md — full ADR with the rejection rationale, root cause, and a scoped follow-up recommendation (fixing check_cut's O(m) lookup is flagged as separate future work, not bundled into this rejected design).
• docs/research/nightly/2026-08-28-mincut-guided-hnsw-repair/ — README (full methodology + raw benchmark output) and gist (standalone write-up).
• docs/adr/INDEX.md — regenerated via node scripts/adr-index.mjs.

Benchmark command & real results

cargo run --release -p ruvector-mincut-repair --bin benchmark

TombstoneOnly:   delete=0.00ms      recall@10=0.9140  repaired_edges=0
BatchRepair(50): delete=0.80ms      recall@10=0.9140  repaired_edges=0
EagerRepair:     delete=0.78ms      recall@10=0.9140  repaired_edges=176
LocalCutGuided:  delete=318024.78ms recall@10=0.9140  repaired_edges=0

Acceptance criteria:
  1. recall gap (Eager - MincutGuided) <= 1.0pp     : +0.00pp   [PASS, but see caveat below]
  2. repaired_edges ratio (Mincut/Eager) <= 0.60    : 0.000     [PASS, but see caveat below]
  3. bookkeeping overhead fraction <= 0.25          : 1.000     [FAIL]

ACCEPTANCE: REJECT

Important honesty caveat (detailed in the ADR/README): at n_delete=3 (reduced from the originally-hypothesized 20% because larger runs didn't complete in bounded time — see "How n_delete Was Chosen" in the README), all three deleted nodes were judged "safe" by the cut-finder, so criteria 1 and 2 pass by inaction, not by successfully triaging fragile nodes. Only criterion 3 (a direct wall-clock measurement, sample-size-independent) carries real evidentiary weight, and it fails decisively and reproducibly across all three runs performed.

Darwin / Flywheel / MetaHarness

npx metaharness is installed and was inspected; npx ruvector harness is not a resolvable package in this environment (recorded, not assumed). No Darwin evolution phase was run — there is no accepted parent candidate to evolve from; the negative result itself is the retained evidence for future nightly runs to avoid rediscovering this dead end.

Security / Governance

No new unsafe code, no I/O, no authorization or provenance surface. N/A for RVF/RVM/MCP/WASM/edge — see ADR and README for the explicit "not evaluated, given rejection" reasoning.

Production recommendation

Do not integrate LocalCutGuidedRepair. Recommended follow-up (separate nightly): fix LocalKCut::check_cut's O(m) edge lookup in ruvector-mincut on its own merits (with its own correctness verification), independent of whether cut-guided HNSW repair is ever revisited.

---

Generated by Claude Code

[ruvnet]

$(cat <<'EOF'

CI status check-in

Tests (core-and-rest) (workflow run 33153656989, job 98791268107) shows cancelled, not failure. Investigated before treating it as this PR's problem.

Why it isn't this PR's: the job logs show it was still mid-compile (60+ crates compiling cleanly, including both crates this PR touches — ruvector-hnsw-repair and ruvector-mincut-repair, with no errors) when GitHub emitted The operation was canceled. exactly 4 hours after start (08:01:25 → 12:01:38 UTC). No test body had started running yet.

This same Workspace CI workflow shows cancelled as its outcome on main itself, repeatedly, independent of what each merge touched — e.g. the 15 most recent main-branch runs of this workflow include at least 11 with conclusion cancelled, spanning unrelated changes (context index, VoI routing, CAMA memory arbitration, Wave-4/5 ADR docs, rustfmt-only fixes, clippy fixes). This is a pre-existing, base-branch-reproducible characteristic of this workflow (the workspace is large enough that a full build+test pass routinely exceeds whatever time/concurrency limit triggers cancellation), not something this PR's two small, additive crates introduced.

Everything else on this PR is green: all other 60 of 61 check runs completed successfully, including Cargo check, Clippy, Clippy (deny warnings), Rustfmt, ADR numbering guard, Workspace membership guard, Security audit, and every other Tests (*) shard (core-and-rest-heavy, core-platform, core-and-rest-wasm, ruvix, vector-index, research-nightly, etc.).

Given the strength of that evidence (11+ independent historical confirmations on main), I'm not spending a re-run here — a re-run of this specific job would itself take multiple hours for a result this pattern already predicts. Flagging as a known, pre-existing infra characteristic rather than a regression from this change.
EOF
)

---

Generated by Claude Code

[ruvnet]

(Correcting a formatting artifact in my previous comment — a stray shell-heredoc wrapper leaked into the posted text. Re-posting clean; no edit tool is available to fix the original.)

CI status check-in

Tests (core-and-rest) (workflow run 33153656989, job 98791268107) shows cancelled, not failure. Investigated before treating it as this PR's problem.

Why it isn't this PR's: the job logs show it was still mid-compile (60+ crates compiling cleanly, including both crates this PR touches — ruvector-hnsw-repair and ruvector-mincut-repair, with no errors) when GitHub emitted The operation was canceled. exactly 4 hours after start (08:01:25 → 12:01:38 UTC). No test body had started running yet.

This same Workspace CI workflow shows cancelled as its outcome on main itself, repeatedly, independent of what each merge touched — e.g. the 15 most recent main-branch runs of this workflow include at least 11 with conclusion cancelled, spanning unrelated changes (context index, VoI routing, CAMA memory arbitration, Wave-4/5 ADR docs, rustfmt-only fixes, clippy fixes). This is a pre-existing, base-branch-reproducible characteristic of this workflow (the workspace is large enough that a full build+test pass routinely exceeds whatever time/concurrency limit triggers cancellation), not something this PR's two small, additive crates introduced.

Everything else on this PR is green: all other 60 of 61 check runs completed successfully, including Cargo check, Clippy, Clippy (deny warnings), Rustfmt, ADR numbering guard, Workspace membership guard, Security audit, and every other Tests (*) shard (core-and-rest-heavy, core-platform, core-and-rest-wasm, ruvix, vector-index, research-nightly, etc.).

Given the strength of that evidence (11+ independent historical confirmations on main), I'm not spending a re-run here — a re-run of this specific job would itself take multiple hours for a result this pattern already predicts. Flagging as a known, pre-existing infra characteristic rather than a regression from this change.

---

Generated by Claude Code

[ruvnet]

Dream evidence review — exact head fd30aa349c2ee8802bf3915ad9422e9bf39c1903

Frozen hypothesis: LocalKCut-guided HNSW deletion repair matches eager-repair recall with less than 25% overhead.

The recorded 3-deletion benchmark rejects the hypothesis: LocalCutGuided takes 318,024.78 ms versus 0.78 ms for EagerRepair, while equal recall is obtained only because cut-guided repair performs no repair. The negative result is valuable and should remain documented.

A separate reachable implementation defect amplifies the result: LocalKCut::check_cut scans/materializes the full edge set for each boundary edge although the graph already maintains indexed lookup. This is now scoped in issue #942 with a seeded correctness benchmark and ≥100× speedup threshold. Fixing that defect must not be used to revive the rejected bounded-locality design without new evidence.

Supply-chain, native, WASM dedup, Clippy, and regression-guard checks pass; the required Workspace CI run was cancelled.

REJECT

[ruvnet]

Thanks for the evidence review — agrees with this PR's own conclusion, so no code changes needed here. One small doc update pushed (96282d9): both the ADR and the nightly README now link directly to issue #942 instead of the vague "separate nightly" phrasing they had, since #942 already scopes exactly the follow-up this ADR recommended (the check_cut O(m) fix, with its own correctness regression and a ≥100x speedup threshold) — including the same "don't use this fix to revive the rejected design" caveat.

To confirm alignment on the CI note: the Workspace CI cancellation was investigated in an earlier comment on this PR — it reproduces identically and repeatedly on main itself (11+ of the last 15 main-branch runs of this workflow), unrelated to any of those merges' content, so it's a pre-existing infra characteristic rather than something this PR's two crates caused.

---

Generated by Claude Code

[ruvnet]

CI status check-in (2)

Tests (core-platform) (workflow run 33238915195, job 99064734875, head 96282d97) failed. Investigated.

What failed: ruvector-replication::replica::tests::test_promotion timed out after 600s — the only failure out of 624 tests in that shard (623 passed).

Why it isn't this PR's: ruvector-replication is a crate this PR does not touch (this PR only adds ruvector-mincut-repair, makes one function pub in ruvector-hnsw-repair, and adds docs). The test body itself is pure synchronous in-memory struct manipulation with no I/O, no sleeps, no network calls (ReplicaSet::new → add_replica ×2 → promote_to_primary → get_primary, three assertions) — it has no code path that should take anywhere near 600 seconds. A hang that long on code shaped like that points to a pre-existing intermittent deadlock (most likely in a shared lock inside ReplicaSet, exposed under nextest's parallel scheduling), not a correctness regression from this diff.

Re-run: attempted rerun_failed_jobs on this workflow run — blocked with 403 This workflow is already running (other shards, e.g. the long-running Tests (core-and-rest), are still in progress). I don't yet have the means to re-run this specific job until the overall run completes; will retry once it does. Keeping this PR watched until it's green — not treating this as someone else's problem to pick up.

---

Generated by Claude Code