RQ-61-EVIDENCE (#1085): R7/R8/R9 — evidence must belong to the release, the release field must match the file, code-existence is not a predicate - #1099
Conversation
…e, the release field must match the file, code-existence is not a predicate Three rules, all found by USING the v0.60 status-evidence gate rather than reasoning about it. Red-first each: all three defective fixtures pass the pre-change checker with 0 failures, then fail on exactly the rule under test and no other. R7 — EVIDENCE MUST BELONG TO THE RELEASE. For every contains:/file: done-when that HOLDS, the oldest first-parent commit introducing the signature (git log -S for contains:, --diff-filter=A for file:) must not be an ancestor of the previous minor's HIGHEST tag (so evidence shipped in a previous PATCH is caught too). The RQ-60-CANARY shape — gate merged 08:50, v0.59.0 tagged 15:19, v0.60 plan written 18:56 — is replayed in a real git fixture; the shipped-in: escape hatch (per-case, version-shaped, reason written beside it, #1090) is the green control exactly as it stands on main. Reliability stated, not silent: no-git/shallow/missing-tag/ uncommitted-signature SKIPS LOUDLY (R7-SKIP warning + a skip count in the machine-read summary), and the CI grep pins '(0 skipped)' with >= 1 archaeology check — a shallow CI checkout already reds the R4 delivery floor before R7 is reached. Live: 4 checks performed, 0 skipped. R8 — THE release: FIELD MUST EQUAL THE FILE'S VERSION. The checker version-gates by PATH; rivet's readiness query reads the FIELD; nothing asserted agreement. The harmful never-hit direction (release: v0.60 parked in a pre-v0.60 file, exempt from every >= v0.60 rule while rivet counts it in v0.60's scope) is the red fixture. THE 6 MEASURED MISMATCHES ARE KEPT under the one now-STATED allowance: a field naming a PATCH of the file's previous minor (vX.(Y-1).Z, Z >= 1) is the parked-patch practice (v0.56.1/v0.56.2 in release-v0.57.yaml) and green; missing, unparseable, ahead-of-path, or behind-without-patch fields are red. No historical release file needed edits. R9 — A contains: INTO CRATE SOURCE IS WEAKER THAN THE GATE. Under a CLAIMING status, a signature whose path is crates/** excluding /tests/ components (crate integration-test dirs are executed by the required Test job and are gate-shaped, like scripts/, coq/ and workflows — measured, every honest live signature already points at one of those) requires a written verified-by saying why code-existence IS the outcome; the better fix is re-pointing at the gate, which is what both #1090 corrections did. The RQ-60-A64IMPORT specimen is the red fixture; both correction forms are green controls. test_4's green control updated to the #1090 corrected done-when — the old fixture carried the exact shape R9 now reds, and R9 caught it. Gates: 42 unit tests OK (7-instance replay intact); live tree 66 artifacts / 0 failures / 4 archaeology checks (0 skipped); 5 mutations each kill >= 1 test with needle counts asserted before mutating (R7 ancestor check, R8 loop, R8 allowance, R9 trigger, R7 skip accounting) and byte-identical restore; claim_check 52/52; test_claim_check 50 OK; ci.yml summary grep extended in the same PR. Artifact flipped implemented with done-when pointing at the R7 replay class in the unit suite (R9 applied to itself) and a written verified-by. Refs #1085, #1090, #1064, #242. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
06db00a to
4442e77
Compare
|
Rebased onto
Nothing here is suspected broken. I verified the real post-merge tree locally before pushing: But a green check is a statement about the tree that was tested, and that tree no longer exists. The commit content is unchanged — only the base moved — so this is a re-run, not a revision. Worth recording because it is the concrete case for the name-conditioned rule: the earlier framing that the queued jobs were "untouched by this change" was true of the diff and false of the tree. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
… idle rust-cpu pool (#1101) The smallest possible first step, and deliberately a NON-REQUIRED job. MEASURED, not argued. Four PRs opened within ~30 minutes each reached EXACTLY 3/9 required contexts within minutes and then stalled for hours. The three that completed were the same three every time — Format, Version Pin Sweep, Rivet Validation — and those are precisely the three that run SELF-HOSTED. The six that stalled are the six on `ubuntu-latest`. Same commits, same moment, same queue; the split is not explained by job cost (Format is not cheaper than Clippy) but entirely by which pool the job asks for. runs-on census: 56 ubuntu-latest · 4 rust-cpu · 2 light · 1 macos org runners: 12 online, 4 busy (OTHER repos), 8 IDLE (7 rust-cpu, 1 light) WHY THIS JOB. `synth-provenance-v1 reconciliation gate` needs only `dtolnay/rust-toolchain` — no apt, no Nix, no `cargo install`, no network package fetch. That is the same shape as `Rivet Validation`, which already runs on `rust-cpu` successfully, so the pool is proven for cargo-shaped work and this move tests scheduling rather than toolchain. WHY NOT A REQUIRED JOB FIRST, even though the required six are what actually blocks. A REQUIRED context naming a label no online runner satisfies does not fail — it NEVER RUNS, and a non-running required check deadlocks every merge in the repo. A non-required job that fails is a red someone can read and revert; a required job that never starts is a repo that cannot merge its own revert. One job per PR, each with its own full green cycle, in the risk order posted to #1062: Claim Check ... checkout + python ONLY .... next, after this cycle Clippy ........ rust-toolchain + cache .... then Kani .......... cargo install ............. then Test / Z3 ..... sudo apt-get .............. only after apt is confirmed Bazel ......... nix + setup-bazel ......... last or never — an environmental failure there looks exactly like a real proof failure THE COST BEING PAID, measured this wave: queue latency converts directly into wasted CI cycles. Two PRs each accumulated green against a base that moved underneath them (#1096 at 8/9, #1099 at 9/9) because GitHub re-runs checks when the BRANCH moves, never when the BASE does. Both had to be rebased and re-run from scratch; #1096 twice. The longer a PR waits, the likelier its green describes a tree that no longer exists. Change is one `runs-on` line: ci.yml ubuntu-latest 50 -> 49, all workflows 56 -> 55. Nothing else moved. Refs #1062, refs #242. Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Closes nothing yet — implements RQ-61-EVIDENCE (#1085): three rules for
scripts/status_evidence_check.py, all found by USING the v0.60 gate rather than reasoning about it.R7 — evidence must belong to the release
For every
contains:/file:done-when that HOLDS, the oldest first-parent commit introducing the signature (git log -S<literal>forcontains:,--diff-filter=Aforfile:) must not be an ancestor of the previous minor's highest tag (vX.(Y-1).*, so "shipped in the previous patch" is caught too; a main-line commit that is an ancestor of a patch tag predates the patch branch point, so a later-tagged patch cannot mis-attribute post-minor work).shipped-in: vX.Z(per-case, version-shaped —shipped-in: earlieris red — reason written beside it). RQ-60-CANARY on main is the live green instance; the gate accepts it.R7-SKIPwarning per artifact plus a skip count in the machine-read summary line. The CI grep (updated in this PR) pins(0 skipped)and ≥ 1 archaeology check, so the loud skip cannot quietly pass in the required job; a shallow CI checkout already reds the R4 delivery floor (28) before R7 is even reached.fetch-depth: 0in the claim-check job provides both history and tags.R7-SKIP: uncommitteduntil committed — the skip contract demonstrated itself).R8 — the
release:field must equal the file's versionThe checker version-gates by PATH; rivet's readiness query reads the FIELD; nothing asserted they agree.
release: v0.60parked inrelease-v0.59.yaml— before this PR it passed every rule (no done-when demanded, path < v0.60) while rivet counts it in v0.60's scope. Now red with a "version-gate bypass" message, and R8 is the only rule that fires on it.vX.(Y-1).Z,Z ≥ 1) is the parked-patch practice — real v0.56.1/v0.56.2 artifacts written up inrelease-v0.57.yamlduring the v0.57 cycle. Correcting the fields would falsify what rivet's release query says actually shipped them, and moving 6 artifacts between shipped-history files churns the graph for zero information; stating the rule is the honest disposition. No historical release file was edited. Missing, unparseable, ahead-of-path, and behind-without-patch fields are all red; the six are replayed as a green fixture.R9 — a
contains:into crate source is weaker than the gate that exercises itUnder a CLAIMING status, a signature whose path is
crates/**excluding/tests/components requires a writtenverified-bysaying why code-existence genuinely IS the outcome — or, better, re-point the signature at the gate (what both #1090 corrections did).scripts/repro/(executed oracle),scripts/test_*.py(required job),coq/**(kernel-checked byverify_proofs),.github/workflows/(the wiring itself). Crate integration-test dirs (crates/*/tests/) are executed by the required Test job (e.g.wcet_bound_gate.rs) and are gate-shaped, so they are deliberately not matched. The only failing class is crate source — the RQ-60-A64IMPORT specimen ("the acceptance number is the deliverable"pinned on a symbol existing).test_4is exactly the shape R9 reds — the fixture was updated to the fix(#1085): two v0.60 artifacts pinned evidence that could not fail on the failure they define #1090 corrected predicate, with a comment saying why.scripts/signature names a gate some CI job actually runs isoracle_wiring_check.py's surface.Red-first / mutation evidence
Pre-change checker on all three defective fixtures: 0 failures each (transcript in lane log). Post-change: each fails on exactly the rule under test and no other (asserted in the tests themselves:
[f for f in fails if not f.startswith("R7 ")] == []etc.).Mutations (needle count asserted before each; byte-identical restore verified):
Files changed, and why
scripts/status_evidence_check.py— R7/R8/R9 + docstring (rules, dispositions, residuals); summary line now carriesN release-scope archaeology checks (N skipped).scripts/test_status_evidence_check.py— 12 new tests (42 total, 7-instance replay intact); fixtures now stamprelease:from the file's own version, matching every artifact on shipped history..github/workflows/ci.yml— the status-evidence summary grep extended for the new line shape and pinning(0 skipped)+ ≥ 1 archaeology check (same PR as the line-shape change, so the grep cannot go stale).artifacts/release-v0.61/RQ-61-EVIDENCE.yaml— status →implementedwith the dispositions written into the description,done-whenpointing at the R7 replay class in the unit suite (R9 applied to itself), and a writtenverified-by. No other release artifact file touched (the R8 disposition required none).Gates
status_evidence_check66 artifacts / 0 failures / 5 archaeology checks (0 skipped) ·test_status_evidence_check42 OK ·claim_check52/52 ·test_claim_check50 OK ·cargo fmt --allclean ·cargo clippy --workspace --all-targets -- -D warningsexit 0.Honest residual
shipped-in's andverified-by's truth are trusted as written; the gate forces them to be written where the release-query reader sees them, it cannot judge them.git log -Soccurrence-count semantics; a literal added, fully removed, and re-added later attributes to its first introduction. The escape hatch covers the pathological case.Refs #1085, #1090, #1064, #242.
🤖 Generated with Claude Code
https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L