fix(deps): remove the phantom loom dependency — declared, never used, shipped anyway - #2574
fix(deps): remove the phantom loom dependency — declared, never used, shipped anyway#2574noahgift wants to merge 2 commits into
Conversation
… shipped anyway `loom` was a declared optional dependency of aprender-profile and NOTHING in the repo has ever used it. MEASURED on origin/main (233bfe1): crates/aprender-profile/Cargo.toml:208 chaos-full = ["chaos-byzantine", "dep:loom", "dep:arbitrary"] crates/aprender-profile/Cargo.toml:222 [dependencies.loom] version = "0.7" optional = true grep -rn --include="*.rs" "loom::" crates/ src/ -> 0 (excluding "Bloom") grep -rn --include="*.rs" "cfg(loom)" crates/ src/ -> 0 Every other `loom` hit in this repo is the string "Bloom" — BloomForCausalLM, a Bloom filter, Bloom's taxonomy. IT WAS ALSO MIS-DECLARED, WHICH IS WHY IT ACTUALLY COST SOMETHING loom's own docs prescribe `[target.'cfg(loom)'.dependencies]`, so the crate is compiled ONLY under `--cfg loom`. This was a plain optional `[dependencies]` entry, so it built in ordinary `--all-features` builds: cargo tree -p aprender-profile --all-features -i loom loom v0.7.2 └── aprender-profile v0.63.0 and aprender-profile sets `[package.metadata.docs.rs] all-features = true`, so docs.rs was building it too. It dragged in generator v0.8.9 (assembly-level stack switching) and scoped-tls for zero benefit. WHY loom IS NOT BEING ADOPTED INSTEAD Investigated properly rather than deleted on sight. loom is an excellent tool for a code shape this repo does not have, and the decisive evidence is a matched pair run on a scratch crate: the SAME lost-update bug is CAUGHT (rc=101) when written with `loom::thread` + loom atomics, and MISSED ENTIRELY (rc=0) when written with `std::thread` + std atomics inside `loom::model`. rayon and tokio use std internally, so loom is structurally blind to essentially all of this repo's concurrency unless every crate is rewritten behind a cfg(loom) shim. loom's own docs state it: "Any code that does not use loom's replacement types is invisible to loom." The census removes the remaining motive. Across aprender-serve's 564 sync sites: 74 AtomicUsize, 21 AtomicBool, 23 mpsc::Sender, 8 Arc<RwLock>, 3 Condvar — and ZERO compare_exchange/fetch_update, ZERO hand-rolled SpinLock/SeqLock/RingBuffer. All 29 `unsafe impl Send/Sync` are FFI marker assertions on CUDA handles, which loom cannot check at all: whether a CUcontext is genuinely Sync is an FFI contract, not an interleaving question. Cost, measured: four threads doing ONE atomic increment each did not finish in 90s unbounded (rc=124); 2.1s at LOOM_MAX_PREEMPTIONS=3. MAX_THREADS is 5 including main, and exceeding it panics rather than skipping. RECORDED DISSENT: if a lock-free structure is ever hand-rolled here — a PagedAttention KV-cache block allocator or a throughput-rewritten batch-scheduler queue are the plausible candidates — loom becomes the right tool immediately, and adopting it in that one module would be cheap. This is "not now, not repo-wide", NOT "never". RECOMMENDED INSTEAD (not done here, needs a pilot): ThreadSanitizer on aprender-serve's existing integration tests. No source changes, sees through rayon AND tokio because it instruments machine code, composes with the suite that already exists. Caveat stated honestly: this is UNVERIFIED — TSan was not run against aprender-serve, it needs nightly, and it can be noisy against CUDA FFI, so pilot it on one CPU-only target before wiring it into CI. VERIFIED AFTER THE REMOVAL: cargo check -p aprender-profile --features chaos-full rc=0 cargo check -p aprender-profile --all-features rc=0 cargo check -p aprender-profile --all-features --locked rc=0 cargo metadata --locked rc=0 cargo tree --workspace --all-features -i loom -> "nothing to print" The `[[package]] loom` block remains in Cargo.lock as an unreferenced leftover; it is in no dependency graph, so it is neither built nor fetched, and it is pruned by the next full resolve. Refs #2566 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scripts/check_facade_compat.sh's BEHAVIOUR row ran
cargo test --quiet --workspace --target-dir "$TD" >/dev/null 2>&1
so a red run emitted exactly one line -- FAIL compat targets do not pass --
and nothing else. That is what #2574 hit on run 32571097931: the compile row
above it passed in 49s, this row failed 16s later, and the log cannot say
whether that was a signature drift, a lockfile problem or a rustc killed on a
runner box under load 204. Reading the verdict required rebuilding by hand on
a DIFFERENT machine state than the one that produced it, which is not evidence.
The output now goes to a file and the failure branch prints it, prefixed. The
success path is unchanged and still silent. One invocation only: re-running
cargo "to get the output" would sample a different machine state than the one
that decided the verdict, which is the same defect wearing a second run.
Mutation-verified in both directions against the real edited lines (extracted
by line range into a harness, not paraphrased), each mutation proved to have
ENGAGED by a diff exiting 1:
unmutated -> rc=0, prints only `ok compat_invoke + compat_probe pass`
assert_eq!(...,999)-> rc=1, prints the panic, the left/right values and
invoke.rs:55
bogus import added -> rc=1, prints E0432 with the rustc span
MEASURED, on the loom question this branch actually raises: removing loom does
NOT break the facade workspace. crates/facades/Cargo.lock names only
aprender-contracts and aprender-contracts-macros among in-tree crates -- no
loom, no aprender-profile -- so the removed dependency is not in that graph.
CI agrees: R5 (`cargo metadata --locked` in crates/facades) and
check_lockfile_current.sh both PASSED on this branch in the same run whose
BEHAVIOUR row went red. `cargo test --quiet --workspace` in crates/facades is
green here on this branch, rc=0, 6 test binaries.
Refs #2574
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR
|
Superseded by #2613, the 0.64.0 integration batch. This PR's commits are merged into Why batched rather than landed individually: one Batching also found four defects that were invisible to every individual PR — most Closing now, deliberately: an open PR that merges first moves #2613's base and forces |
loomwas a declared optional dependency of aprender-profile and NOTHING in therepo has ever used it.
MEASURED on origin/main (233bfe1):
Every other
loomhit in this repo is the string "Bloom" — BloomForCausalLM, aBloom filter, Bloom's taxonomy.
IT WAS ALSO MIS-DECLARED, WHICH IS WHY IT ACTUALLY COST SOMETHING
loom's own docs prescribe
[target.'cfg(loom)'.dependencies], so the crate iscompiled ONLY under
--cfg loom. This was a plain optional[dependencies]entry, so it built in ordinary
--all-featuresbuilds:and aprender-profile sets
[package.metadata.docs.rs] all-features = true, sodocs.rs was building it too. It dragged in generator v0.8.9 (assembly-level
stack switching) and scoped-tls for zero benefit.
WHY loom IS NOT BEING ADOPTED INSTEAD
Investigated properly rather than deleted on sight. loom is an excellent tool for
a code shape this repo does not have, and the decisive evidence is a matched pair
run on a scratch crate: the SAME lost-update bug is CAUGHT (rc=101) when written
with
loom::thread+ loom atomics, and MISSED ENTIRELY (rc=0) when written withstd::thread+ std atomics insideloom::model. rayon and tokio use stdinternally, so loom is structurally blind to essentially all of this repo's
concurrency unless every crate is rewritten behind a cfg(loom) shim. loom's own
docs state it: "Any code that does not use loom's replacement types is invisible
to loom."
The census removes the remaining motive. Across aprender-serve's 564 sync sites:
74 AtomicUsize, 21 AtomicBool, 23 mpsc::Sender, 8 Arc, 3 Condvar — and
ZERO compare_exchange/fetch_update, ZERO hand-rolled SpinLock/SeqLock/RingBuffer.
All 29
unsafe impl Send/Syncare FFI marker assertions on CUDA handles, whichloom cannot check at all: whether a CUcontext is genuinely Sync is an FFI
contract, not an interleaving question.
Cost, measured: four threads doing ONE atomic increment each did not finish in
90s unbounded (rc=124); 2.1s at LOOM_MAX_PREEMPTIONS=3. MAX_THREADS is 5
including main, and exceeding it panics rather than skipping.
RECORDED DISSENT: if a lock-free structure is ever hand-rolled here — a
PagedAttention KV-cache block allocator or a throughput-rewritten batch-scheduler
queue are the plausible candidates — loom becomes the right tool immediately, and
adopting it in that one module would be cheap. This is "not now, not repo-wide",
NOT "never".
RECOMMENDED INSTEAD (not done here, needs a pilot): ThreadSanitizer on
aprender-serve's existing integration tests. No source changes, sees through
rayon AND tokio because it instruments machine code, composes with the suite that
already exists. Caveat stated honestly: this is UNVERIFIED — TSan was not run
against aprender-serve, it needs nightly, and it can be noisy against CUDA FFI,
so pilot it on one CPU-only target before wiring it into CI.
VERIFIED AFTER THE REMOVAL:
cargo check -p aprender-profile --features chaos-full rc=0
cargo check -p aprender-profile --all-features rc=0
cargo check -p aprender-profile --all-features --locked rc=0
cargo metadata --locked rc=0
cargo tree --workspace --all-features -i loom -> "nothing to print"
The
[[package]] loomblock remains in Cargo.lock as an unreferenced leftover;it is in no dependency graph, so it is neither built nor fetched, and it is
pruned by the next full resolve.
Refs #2566
Co-Authored-By: Claude Opus 5 noreply@anthropic.com