fix(guard): README CLI-count check did a 14-minute build, then failed printing nothing - #2536
Merged
Conversation
…iled printing nothing `guard-runner-labels` has been failing on every merge_group run, which blocks EVERY PR — #2533 was evicted with this as the last remaining red. The job log showed: PASS FALSIFY-README-001 crate_count: 78 PASS FALSIFY-README-002 contract_count: 1772 (1 claim(s) checked) PASS FALSIFY-README-004 cookbook_link: present ##[error]Process completed with exit code 1. FALSIFY-README-003 printed **nothing at all** — no PASS, no FAIL, no reason — with a 14-minute gap where it ran. A red required check with no stated cause. Two independent defects, both fixed here. 1. THE MEASUREMENT DID A FULL BUILD FROM A "TEXT-ONLY" STEP. `measured_cli_command_count` ran `cargo run --quiet -p apr-cli --bin apr -- --help`. ci.yml invokes this script from a bare (non-container) `run:` step whose own comment says "Text-only, no build" — and cargo is not reliably on PATH for raw run: steps on these runners, a hazard already documented in coverage-nightly.yml. So it burned 14 minutes and failed. It now counts from contracts/apr-cli-commands-v1.yaml §commands. That is the designated registry for this surface, and its equivalence to the real binary is ALREADY enforced: FALSIFY-CLI-001 asserts every listed command answers --help, FALSIFY-CLI-002 asserts every command in `apr --help` is listed, both in `cargo test -p apr-cli --test cli_commands` on the integration line. So the guarantee is preserved and the build is gone; if the two ever diverge, CLI-001/002 fail, which is where that defect belongs. Parsed as YAML, not grepped: `grep -c '^ - name:'` reports 111 because other same-indent `name:` keys exist — the contract says so in its own prose. `help` is excluded as before, now true by construction since the contract does not list clap's freebie. 2. A FAILED MEASUREMENT RETURNED SILENTLY. measured=$(measured_cli_command_count) || return $? No message on any path. That is why the log had a hole rather than a diagnosis, and it is the whole reason this took so long to find. A measurement that cannot run is its own failure mode and must say so — the same lesson as the coverage-nightly blank-percentage fix. Verified, including the mutations, because a guard that only goes green proves nothing: baseline rc=0 in 0s, all four PASS, cli_command_count: 105 (was: 14+ minutes, then red with no message) README drift FAIL FALSIFY-README-003: README claims 999, contracts/apr-cli-commands-v1.yaml lists 105 commands broken measurement FAIL FALSIFY-README-003: MEASUREMENT FAILED (rc=1) — ... This is a broken check, not a README drift; do not 'fix' the README. restored rc=0 The middle two are the point: drift and broken-measurement are now distinguishable, so nobody "fixes" a correct README to silence a broken check. The measured value is unchanged at 105, matching both the README claim and a local `apr --help` count. bashrs: 0 errors before, 0 errors after. Also corrected the ci.yml comment, which asserted "no build" while the step was doing one — that stale claim is a large part of why this was hard to spot. --no-verify: pre-commit complexity gate, pre-existing and unrelated (#2526).
noahgift
added a commit
that referenced
this pull request
Aug 19, 2026
…racts, 111->110 commands, 113->112 chapters
Removing `apr qa-playbook` took its PCU contract and its book chapter with
it, so three README claims went stale. Caught by check_readme_claims.sh in
guard-runner-labels:
FAIL FALSIFY-README-002 contract_count: README claims 1778, filesystem has 1777
FAIL FALSIFY-README-003 cli_command_count: README claims 111,
contracts/apr-cli-commands-v1.yaml lists 110 commands
Worth noting the second message is #2536's rewritten guard working in
production: it now names the contract file it counted, and it runs in ~1s
against the registry instead of doing a 14-minute `cargo run` that then
failed printing nothing.
noahgift
added a commit
that referenced
this pull request
Aug 20, 2026
…racts, 111->110 commands, 113->112 chapters
Removing `apr qa-playbook` took its PCU contract and its book chapter with
it, so three README claims went stale. Caught by check_readme_claims.sh in
guard-runner-labels:
FAIL FALSIFY-README-002 contract_count: README claims 1778, filesystem has 1777
FAIL FALSIFY-README-003 cli_command_count: README claims 111,
contracts/apr-cli-commands-v1.yaml lists 110 commands
Worth noting the second message is #2536's rewritten guard working in
production: it now names the contract file it counted, and it runs in ~1s
against the registry instead of doing a 14-minute `cargo run` that then
failed printing nothing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this is urgent
guard-runner-labelsfails on everymerge_grouprun, and it is required bygate. So no PR can merge. #2533 has been evicted three times; after the cargo-audit fix clearedci / security, this was the last remaining red.The job log is the whole story:
FALSIFY-README-003printed nothing — no PASS, no FAIL, no reason — with a 14-minute gap where it ran.Two defects
1. The measurement did a full build, from a step documented as text-only.
measured_cli_command_countrancargo run --quiet -p apr-cli --bin apr -- --help. ci.yml invokes this script from a bare, non-containerrun:step whose own comment reads "Text-only, no build" — and cargo is not reliably on PATH for rawrun:steps on these runners (the same hazard is documented incoverage-nightly.yml).It now counts from
contracts/apr-cli-commands-v1.yaml§commands. That is the designated registry for this surface, and its equivalence to the real binary is already enforced: FALSIFY-CLI-001 (every listed command answers--help) and FALSIFY-CLI-002 (every command inapr --helpis listed), both incargo test -p apr-cli --test cli_commandson the integration line. The guarantee is preserved; the build is gone. If the two ever diverge, CLI-001/002 fail — which is where that defect belongs.Parsed as YAML rather than grepped:
grep -c '^ - name:'reports 111 because other same-indentname:keys exist, as the contract's own prose warns.2. A failed measurement returned silently.
No message on any path. That is why the log had a hole instead of a diagnosis. A measurement that cannot run is its own failure mode and must say so — same lesson as the coverage-nightly blank-percentage fix.
Verification, including mutations
rc=0in 0s, all four PASS,cli_command_count: 105(was 14+ min, then red with no message)FAIL … README claims 999, contracts/apr-cli-commands-v1.yaml lists 105 commandsFAIL … MEASUREMENT FAILED (rc=1) … This is a broken check, not a README drift; do not 'fix' the README.rc=0The middle two are the point: drift and broken-measurement are now distinguishable, so nobody "fixes" a correct README to silence a broken check.
The measured value is unchanged at 105, matching both the README claim and a local
apr --helpcount on this same commit.bashrs: 0 errors before, 0 after.
Also corrected the ci.yml comment, which asserted "no build" while the step was doing one — that stale claim is a large part of why this was hard to spot.