FEAT-088: a forgotten safety goal can't hide behind the undeveloped diamond - #163
Merged
Conversation
… diamond `rivet coverage` prints goal-has-support 2/5 = 40% and cannot tell a goal DECLARED `undeveloped: true` -- GSN's diamond, an honest declared incompleteness -- from one somebody forgot. J-004 investigated this on 2026-08-25 and got it right: all three uncovered goals are deliberate, and adding links would be cosmetic. That investigation is correct, and it is prose. Prose is exactly what the next forgotten goal hides behind -- add a goal with no evidence and it reads like the three that were reasoned about. J-004 says so itself: "a genuinely forgotten goal would hide among the declared ones." This makes that sentence mechanical. Two directions, and the second is the one that matters: 1. no supporting safety-solution -> MUST carry `undeveloped: true` 2. carries `undeveloped: true` -> MUST be named by a safety-justification (1) alone is satisfiable by typing the flag, so a gate checking only it catches the mistake nobody was going to make. (2) is the shape a forgotten goal takes once someone notices the number and quiets it. The checker cross-checks its population against `rivet list` and fails on disagreement -- not padding: G-005 lives in roadmap-2.0.yaml, not safety-case.yaml, so reading the obvious single file finds 4 of 5 and silently omits the goal carrying asil: D. It must not be able to have the bug it prevents. Fails CLOSED when PyYAML is missing instead of skipping (scry#141's failure mode was a gate that reported green while running nothing). MUTATION-CHECKED against the REAL repo, both directions: scan only safety-case.yaml -> "rivet sees 5 ... this checker sees 4", exit 1 redact J-004 links + prose -> exactly 3 violations (G-003/4/5), exit 1 restored -> PASS PyYAML shadowed on PYTHONPATH -> exit 1 on BOTH the gate and its self-test Self-test covers 6 cases including the subtle one and runs BEFORE the real check in CI, mirroring tools/check-gate-coverage.py. DOES NOT make the 40% meaningful -- rivet still counts declared-undeveloped as uncovered and still prints 40%. This closes one direction only. Stated in the artifact residual so the next sweep doesn't read "gated" as "resolved", which is the re-litigation J-004 exists to stop. G-003/G-004/G-005 remain deliberately undeveloped; this does not develop them. rivet=0 claim-check=0 gate-coverage=0 undeveloped-goals=0 fmt=0. Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
📐 rivet artifact deltaPR: #163 Base SHA: Validationhead — `rivet validate` resultbase — `rivet validate` result (for comparison)Artifact stats
full stats — headDiff (base → head)AADL model — headPosted by the |
avrabe
added a commit
that referenced
this pull request
Aug 27, 2026
… 7 -> 4 `implemented`/`proposed` count as NOT-yet-verified in the release gate, so a shipped, green feature left there blocks its release indefinitely. These three have shipped AND been verified, so leaving them proposed understates the release just as promoting them early would overstate it. Evidence for each, grounded rather than remembered: FEAT-087 (#162, 11/11 CI green) -- 3 oracles re-run on main just now: the tier-discrimination test, the tier-2 independence pin, and the module-scoped decision. Mutation-checked both directions when landed. FEAT-088 (#163, 11/11 CI green) -- the gate's --self-test PASSES and the real check PASSES on main, and it is wired into ci.yml (self-test before the real check). Mutation-checked against the real repo when landed. FEAT-090 (#166, 11/11 CI green) -- 5 oracles re-run on main just now, including the polarity test that dies only under the `&=` -> `|=` mutant. NOT promoted, deliberately: FEAT-089 -- FILED, not built. The non-zero-fact work does not exist yet; its own AC#1 requires a red test that is still red by design. FEAT-064 -- AC3 closed by measurement in #164, but AC1 remains FALSIFIED (repaired by FEAT-077 only for uniquely-named functions). Closing one AC does not clear the artifact, and REQ-020 stays blocked behind it. FEAT-057, FEAT-065, REQ-021 -- unbuilt. rivet=0 claim-check=0 fmt=0. Refs: FEAT-087 Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
avrabe
added a commit
that referenced
this pull request
Aug 27, 2026
… 7 -> 4 `implemented`/`proposed` count as NOT-yet-verified in the release gate, so a shipped, green feature left there blocks its release indefinitely. These three have shipped AND been verified, so leaving them proposed understates the release just as promoting them early would overstate it. Evidence for each, grounded rather than remembered: FEAT-087 (#162, 11/11 CI green) -- 3 oracles re-run on main just now: the tier-discrimination test, the tier-2 independence pin, and the module-scoped decision. Mutation-checked both directions when landed. FEAT-088 (#163, 11/11 CI green) -- the gate's --self-test PASSES and the real check PASSES on main, and it is wired into ci.yml (self-test before the real check). Mutation-checked against the real repo when landed. FEAT-090 (#166, 11/11 CI green) -- 5 oracles re-run on main just now, including the polarity test that dies only under the `&=` -> `|=` mutant. NOT promoted, deliberately: FEAT-089 -- FILED, not built. The non-zero-fact work does not exist yet; its own AC#1 requires a red test that is still red by design. FEAT-064 -- AC3 closed by measurement in #164, but AC1 remains FALSIFIED (repaired by FEAT-077 only for uniquely-named functions). Closing one AC does not clear the artifact, and REQ-020 stays blocked behind it. FEAT-057, FEAT-065, REQ-021 -- unbuilt. rivet=0 claim-check=0 fmt=0. Refs: FEAT-087 Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
avrabe
added a commit
that referenced
this pull request
Aug 27, 2026
… 7 -> 4 `implemented`/`proposed` count as NOT-yet-verified in the release gate, so a shipped, green feature left there blocks its release indefinitely. These three have shipped AND been verified, so leaving them proposed understates the release just as promoting them early would overstate it. Evidence for each, grounded rather than remembered: FEAT-087 (#162, 11/11 CI green) -- 3 oracles re-run on main just now: the tier-discrimination test, the tier-2 independence pin, and the module-scoped decision. Mutation-checked both directions when landed. FEAT-088 (#163, 11/11 CI green) -- the gate's --self-test PASSES and the real check PASSES on main, and it is wired into ci.yml (self-test before the real check). Mutation-checked against the real repo when landed. FEAT-090 (#166, 11/11 CI green) -- 5 oracles re-run on main just now, including the polarity test that dies only under the `&=` -> `|=` mutant. NOT promoted, deliberately: FEAT-089 -- FILED, not built. The non-zero-fact work does not exist yet; its own AC#1 requires a red test that is still red by design. FEAT-064 -- AC3 closed by measurement in #164, but AC1 remains FALSIFIED (repaired by FEAT-077 only for uniquely-named functions). Closing one AC does not clear the artifact, and REQ-020 stays blocked behind it. FEAT-057, FEAT-065, REQ-021 -- unbuilt. rivet=0 claim-check=0 fmt=0. Refs: FEAT-087 Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
avrabe
added a commit
that referenced
this pull request
Aug 27, 2026
… 7 -> 4 (#173) `implemented`/`proposed` count as NOT-yet-verified in the release gate, so a shipped, green feature left there blocks its release indefinitely. These three have shipped AND been verified, so leaving them proposed understates the release just as promoting them early would overstate it. Evidence for each, grounded rather than remembered: FEAT-087 (#162, 11/11 CI green) -- 3 oracles re-run on main just now: the tier-discrimination test, the tier-2 independence pin, and the module-scoped decision. Mutation-checked both directions when landed. FEAT-088 (#163, 11/11 CI green) -- the gate's --self-test PASSES and the real check PASSES on main, and it is wired into ci.yml (self-test before the real check). Mutation-checked against the real repo when landed. FEAT-090 (#166, 11/11 CI green) -- 5 oracles re-run on main just now, including the polarity test that dies only under the `&=` -> `|=` mutant. NOT promoted, deliberately: FEAT-089 -- FILED, not built. The non-zero-fact work does not exist yet; its own AC#1 requires a red test that is still red by design. FEAT-064 -- AC3 closed by measurement in #164, but AC1 remains FALSIFIED (repaired by FEAT-077 only for uniquely-named functions). Closing one AC does not clear the artifact, and REQ-020 stays blocked behind it. FEAT-057, FEAT-065, REQ-021 -- unbuilt. rivet=0 claim-check=0 fmt=0. Refs: FEAT-087 Claude-Session: https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
rivet coverageprintsgoal-has-support 2/5 = 40.0%and cannot distinguisha goal deliberately marked
undeveloped: true— GSN's diamond, an honestdeclared incompleteness — from one somebody forgot. Both render as uncovered.
J-004 investigated this on 2026-08-25 and got it right: G-003 / G-004 / G-005 are
deliberately undeveloped, and closing the number with links would be cosmetic.
That investigation is correct, and it is prose. Prose is exactly what the next
forgotten goal hides behind — add a goal with no evidence and it reads like the
three that were reasoned about. J-004 says so itself:
This makes that sentence mechanical.
The gate
safety-solution→ must carryundeveloped: trueundeveloped: true→ must be named by asafety-justificationRule 1 alone is satisfiable by typing the flag, so a gate checking only it catches
the mistake nobody was going to make. Rule 2 is the one that matters.
A justification naming the goal only in prose (a
rationalefield, no typed link)counts — J-004's own shape must not be flagged.
Why the cross-check exists
The checker cross-checks its own population against
rivet listand fails ondisagreement. Not defensive padding: G-005 lives in
roadmap-2.0.yaml, notsafety-case.yaml, so a checker reading the obvious single file finds 4 of 5goals, reports clean, and silently omits the one carrying
asil: D.The checker must not be able to have the bug it exists to prevent.
It also fails closed when PyYAML is missing rather than skipping — scry#141's
failure mode was a gate reporting green while running nothing.
Mutation-checked against the real repo
safety-case.yamlrivet sees 5 ... this checker sees 4→ exit 1PYTHONPATHSelf-test covers 6 cases including the subtle one, and runs before the real
check in CI — the
tools/check-gate-coverage.pypattern.What this does NOT do
It does not make the 40% meaningful.
rivet coveragestill countsdeclared-undeveloped goals as uncovered and will still print
goal-has-support 2/5 40.0%. J-004 notes the figure misleads in bothdirections; this closes exactly one — forgotten hiding among declared. The other
needs rivet itself to express
undeveloped, already reported upstream.G-003 / G-004 / G-005 remain deliberately undeveloped. This does not develop
them, and the residual records what developing each would actually require.
rivet=0 claim-check=0 gate-coverage=0 undeveloped-goals=0 fmt=0gh pr checksby hand.🤖 Generated with Claude Code
https://claude.ai/code/session_01KkNzkNYzPh7366DkNijeNc