Skip to content

fix(deps): remove the phantom loom dependency — declared, never used, shipped anyway - #2574

Closed
noahgift wants to merge 2 commits into
mainfrom
fix/remove-phantom-loom-dep
Closed

fix(deps): remove the phantom loom dependency — declared, never used, shipped anyway#2574
noahgift wants to merge 2 commits into
mainfrom
fix/remove-phantom-loom-dep

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

loom was a declared optional dependency of aprender-profile and NOTHING in the
repo has ever used it.

MEASURED on origin/main (233bfe1):

crates/aprender-profile/Cargo.toml:208
    chaos-full = ["chaos-byzantine", "dep:loom", "dep:arbitrary"]
crates/aprender-profile/Cargo.toml:222
    [dependencies.loom] version = "0.7" optional = true

grep -rn --include="*.rs" "loom::"    crates/ src/  ->  0   (excluding "Bloom")
grep -rn --include="*.rs" "cfg(loom)" crates/ src/  ->  0

Every other loom hit in this repo is the string "Bloom" — BloomForCausalLM, a
Bloom filter, Bloom's taxonomy.

IT WAS ALSO MIS-DECLARED, WHICH IS WHY IT ACTUALLY COST SOMETHING

loom's own docs prescribe [target.'cfg(loom)'.dependencies], so the crate is
compiled ONLY under --cfg loom. This was a plain optional [dependencies]
entry, so it built in ordinary --all-features builds:

cargo tree -p aprender-profile --all-features -i loom
  loom v0.7.2
  └── aprender-profile v0.63.0

and aprender-profile sets [package.metadata.docs.rs] all-features = true, so
docs.rs was building it too. It dragged in generator v0.8.9 (assembly-level
stack switching) and scoped-tls for zero benefit.

WHY loom IS NOT BEING ADOPTED INSTEAD

Investigated properly rather than deleted on sight. loom is an excellent tool for
a code shape this repo does not have, and the decisive evidence is a matched pair
run on a scratch crate: the SAME lost-update bug is CAUGHT (rc=101) when written
with loom::thread + loom atomics, and MISSED ENTIRELY (rc=0) when written with
std::thread + std atomics inside loom::model. rayon and tokio use std
internally, so loom is structurally blind to essentially all of this repo's
concurrency unless every crate is rewritten behind a cfg(loom) shim. loom's own
docs state it: "Any code that does not use loom's replacement types is invisible
to loom."

The census removes the remaining motive. Across aprender-serve's 564 sync sites:
74 AtomicUsize, 21 AtomicBool, 23 mpsc::Sender, 8 Arc, 3 Condvar — and
ZERO compare_exchange/fetch_update, ZERO hand-rolled SpinLock/SeqLock/RingBuffer.
All 29 unsafe impl Send/Sync are FFI marker assertions on CUDA handles, which
loom cannot check at all: whether a CUcontext is genuinely Sync is an FFI
contract, not an interleaving question.

Cost, measured: four threads doing ONE atomic increment each did not finish in
90s unbounded (rc=124); 2.1s at LOOM_MAX_PREEMPTIONS=3. MAX_THREADS is 5
including main, and exceeding it panics rather than skipping.

RECORDED DISSENT: if a lock-free structure is ever hand-rolled here — a
PagedAttention KV-cache block allocator or a throughput-rewritten batch-scheduler
queue are the plausible candidates — loom becomes the right tool immediately, and
adopting it in that one module would be cheap. This is "not now, not repo-wide",
NOT "never".

RECOMMENDED INSTEAD (not done here, needs a pilot): ThreadSanitizer on
aprender-serve's existing integration tests. No source changes, sees through
rayon AND tokio because it instruments machine code, composes with the suite that
already exists. Caveat stated honestly: this is UNVERIFIED — TSan was not run
against aprender-serve, it needs nightly, and it can be noisy against CUDA FFI,
so pilot it on one CPU-only target before wiring it into CI.

VERIFIED AFTER THE REMOVAL:
cargo check -p aprender-profile --features chaos-full rc=0
cargo check -p aprender-profile --all-features rc=0
cargo check -p aprender-profile --all-features --locked rc=0
cargo metadata --locked rc=0
cargo tree --workspace --all-features -i loom -> "nothing to print"

The [[package]] loom block remains in Cargo.lock as an unreferenced leftover;
it is in no dependency graph, so it is neither built nor fetched, and it is
pruned by the next full resolve.

Refs #2566

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

noahgift and others added 2 commits August 22, 2026 13:44
… shipped anyway

`loom` was a declared optional dependency of aprender-profile and NOTHING in the
repo has ever used it.

MEASURED on origin/main (233bfe1):

    crates/aprender-profile/Cargo.toml:208
        chaos-full = ["chaos-byzantine", "dep:loom", "dep:arbitrary"]
    crates/aprender-profile/Cargo.toml:222
        [dependencies.loom] version = "0.7" optional = true

    grep -rn --include="*.rs" "loom::"    crates/ src/  ->  0   (excluding "Bloom")
    grep -rn --include="*.rs" "cfg(loom)" crates/ src/  ->  0

Every other `loom` hit in this repo is the string "Bloom" — BloomForCausalLM, a
Bloom filter, Bloom's taxonomy.

IT WAS ALSO MIS-DECLARED, WHICH IS WHY IT ACTUALLY COST SOMETHING

loom's own docs prescribe `[target.'cfg(loom)'.dependencies]`, so the crate is
compiled ONLY under `--cfg loom`. This was a plain optional `[dependencies]`
entry, so it built in ordinary `--all-features` builds:

    cargo tree -p aprender-profile --all-features -i loom
      loom v0.7.2
      └── aprender-profile v0.63.0

and aprender-profile sets `[package.metadata.docs.rs] all-features = true`, so
docs.rs was building it too. It dragged in generator v0.8.9 (assembly-level
stack switching) and scoped-tls for zero benefit.

WHY loom IS NOT BEING ADOPTED INSTEAD

Investigated properly rather than deleted on sight. loom is an excellent tool for
a code shape this repo does not have, and the decisive evidence is a matched pair
run on a scratch crate: the SAME lost-update bug is CAUGHT (rc=101) when written
with `loom::thread` + loom atomics, and MISSED ENTIRELY (rc=0) when written with
`std::thread` + std atomics inside `loom::model`. rayon and tokio use std
internally, so loom is structurally blind to essentially all of this repo's
concurrency unless every crate is rewritten behind a cfg(loom) shim. loom's own
docs state it: "Any code that does not use loom's replacement types is invisible
to loom."

The census removes the remaining motive. Across aprender-serve's 564 sync sites:
74 AtomicUsize, 21 AtomicBool, 23 mpsc::Sender, 8 Arc<RwLock>, 3 Condvar — and
ZERO compare_exchange/fetch_update, ZERO hand-rolled SpinLock/SeqLock/RingBuffer.
All 29 `unsafe impl Send/Sync` are FFI marker assertions on CUDA handles, which
loom cannot check at all: whether a CUcontext is genuinely Sync is an FFI
contract, not an interleaving question.

Cost, measured: four threads doing ONE atomic increment each did not finish in
90s unbounded (rc=124); 2.1s at LOOM_MAX_PREEMPTIONS=3. MAX_THREADS is 5
including main, and exceeding it panics rather than skipping.

RECORDED DISSENT: if a lock-free structure is ever hand-rolled here — a
PagedAttention KV-cache block allocator or a throughput-rewritten batch-scheduler
queue are the plausible candidates — loom becomes the right tool immediately, and
adopting it in that one module would be cheap. This is "not now, not repo-wide",
NOT "never".

RECOMMENDED INSTEAD (not done here, needs a pilot): ThreadSanitizer on
aprender-serve's existing integration tests. No source changes, sees through
rayon AND tokio because it instruments machine code, composes with the suite that
already exists. Caveat stated honestly: this is UNVERIFIED — TSan was not run
against aprender-serve, it needs nightly, and it can be noisy against CUDA FFI,
so pilot it on one CPU-only target before wiring it into CI.

VERIFIED AFTER THE REMOVAL:
    cargo check -p aprender-profile --features chaos-full        rc=0
    cargo check -p aprender-profile --all-features               rc=0
    cargo check -p aprender-profile --all-features --locked      rc=0
    cargo metadata --locked                                      rc=0
    cargo tree --workspace --all-features -i loom   ->  "nothing to print"

The `[[package]] loom` block remains in Cargo.lock as an unreferenced leftover;
it is in no dependency graph, so it is neither built nor fetched, and it is
pruned by the next full resolve.

Refs #2566

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scripts/check_facade_compat.sh's BEHAVIOUR row ran

    cargo test --quiet --workspace --target-dir "$TD" >/dev/null 2>&1

so a red run emitted exactly one line -- FAIL compat targets do not pass --
and nothing else. That is what #2574 hit on run 32571097931: the compile row
above it passed in 49s, this row failed 16s later, and the log cannot say
whether that was a signature drift, a lockfile problem or a rustc killed on a
runner box under load 204. Reading the verdict required rebuilding by hand on
a DIFFERENT machine state than the one that produced it, which is not evidence.

The output now goes to a file and the failure branch prints it, prefixed. The
success path is unchanged and still silent. One invocation only: re-running
cargo "to get the output" would sample a different machine state than the one
that decided the verdict, which is the same defect wearing a second run.

Mutation-verified in both directions against the real edited lines (extracted
by line range into a harness, not paraphrased), each mutation proved to have
ENGAGED by a diff exiting 1:

  unmutated          -> rc=0, prints only `ok    compat_invoke + compat_probe pass`
  assert_eq!(...,999)-> rc=1, prints the panic, the left/right values and
                        invoke.rs:55
  bogus import added -> rc=1, prints E0432 with the rustc span

MEASURED, on the loom question this branch actually raises: removing loom does
NOT break the facade workspace. crates/facades/Cargo.lock names only
aprender-contracts and aprender-contracts-macros among in-tree crates -- no
loom, no aprender-profile -- so the removed dependency is not in that graph.
CI agrees: R5 (`cargo metadata --locked` in crates/facades) and
check_lockfile_current.sh both PASSED on this branch in the same run whose
BEHAVIOUR row went red. `cargo test --quiet --workspace` in crates/facades is
green here on this branch, rc=0, 6 test binaries.

Refs #2574

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR
@noahgift

Copy link
Copy Markdown
Contributor Author

Superseded by #2613, the 0.64.0 integration batch.

This PR's commits are merged into batch/release-0-64-0 verbatim (--no-ff, never rebased),
and #2613's body carries the full provenance table — PR number, branch, merged head SHA, and
the issues each closes — so the detail survives the squash.

Why batched rather than landed individually: one workspace-test run is ~58 minutes on a
shared box. Thirteen PRs cost thirteen runs whether they go serially or in parallel; one
integration branch costs one. The same approach landed 24 branches previously.

Batching also found four defects that were invisible to every individual PR — most
notably the README contract count: #2548, #2549 and #2587 each add exactly one contract, each
is individually correct at 1779, and three +1s collide on one literal (correct value 1781).
That is the exact class that killed the previous batch.

Closing now, deliberately: an open PR that merges first moves #2613's base and forces
another full run. This is reversible and the branch is untouched — reopen if #2613 is
abandoned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant