[kyle-sexton]

Requested by Kyle · project thread

Closes #6386

Summary

Before: rule-throw-only-oracle's remedy had two positive pins in cant-fail-scan.test.sh and no negative one. The detector-findings adopter row for testing:audit read 7 of 13 and listed the rule as owed its negative.

After: the test also checks that this remedy is not the weak-oracle remedy ("Assert the value the code should produce"). The adopter row reads 8 of 13, with all eight report-only rules pinned positive and negative.

Fix

• plugins/testing/skills/audit/scripts/cant-fail-scan.test.sh: one assert_not_contains after the two existing throw-only pins.
• docs/conventions/detector-findings/README.md: the testing:audit row's tally changes from 7 of 13 to 8 of 13, in both the summary cell and the bold "Remedy assertions" sentence. The "owed its negative" wording is gone.
• docs/conventions/detector-findings/CHANGELOG.md: a [3.7.1] patch entry (docs-only). That file defines a new adopter tally as no change to a producer-owned rule.
• No plugin changelog fragment: scripts/check-changelog-fragments.sh --check-required origin/main reports that the change touches no fragment-mode plugin's shipped files.

Verification

• bash plugins/testing/skills/audit/scripts/cant-fail-scan.test.sh: the new assertion passes, PASS: throw-only remedy is not the weak-oracle remedy. The run was stopped partway when the overnight run was wound down: 1,283 PASS and 0 FAIL at that point, not a completed suite. CI's test-bash lane, which runs once the PR is ready, is the full signal.
• scripts/check-changelog-fragments.sh --check-required origin/main: pass.
• markdownlint-cli2 on the two changed docs: 0 issues. No em dashes added.
• Not run: scripts/affected-tests.sh --base origin/main --run.

Related

• feat(testing): judge Reqnroll [Then] steps as tests in the cannot-fail scan #7049 (Reqnroll [Then] steps in the cannot-fail scan) also edits cant-fail-scan.test.sh, in different lines.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L8KdG7fWg2YrErVYWtohmR

---

Generated by Claude Code

[kyle-sexton]

ci-status is red because this PR is a draft, not because of the change. On a draft every lint and test lane skips (lint-repo, test-bash, test-node, test-python, check-skills, check-plugins), and the aggregate counts a skip as not passing. See .github/workflows/pr-require-checks.yml:507-513.

The lanes run once the PR is marked ready. That flip waits for Kyle, because it also starts the review lanes. Until then, the local runs listed under Verification in the PR body are the signal.

---

Generated by Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-13T13:21:43.542213Z	4182a80	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[claude]

Claude finished @kyle-sexton's task in 49s —— View job

---

Security review complete

• Gather context (PR diff, changed files)
• Apply skip gate
• Review changed files for security-relevant surface
• Check for instruction-surface deletions
• Post final findings

No security-relevant surface. All four changed files are test/docs:

• plugins/testing/skills/audit/scripts/cant-fail-scan.test.sh — adds one assert_not_contains call to an existing bash test, pinning that the rule-throw-only-oracle remedy string is not the rule-weak-oracle remedy string. No new code path, no execution of untrusted input, no injection surface.
• docs/conventions/detector-findings/README.md — updates a pinned-assertion tally (7/13 → 8/13) and removes wording that is no longer true now that the negative assertion exists. This closes a documented remedy-assertion gap rather than removing or softening a guardrail, so the instruction-surface-deletion check in this skill's criteria does not apply.
• docs/conventions/detector-findings/CHANGELOG.md — a docs-only patch entry recording the same tally change.
• .changes/testing/test-6386-throw-only-negative-remedy-28e5f6a7.md — a bump: none changelog fragment, no shipped-code impact.

No secrets, credentials, command/path/template injection, or authorization logic touched by this diff. Nothing to flag.

claude-lane-summary: lane=claude-security-review sha=4182a80dd2d98385524d06c8fa213815e660dc39 run=38403720777

[claude]

Claude finished @kyle-sexton's task in 2m 33s —— View job

---

Code review complete

• Gather context (PR body, diff, custom instructions)
• Invoke /review:code-review and follow its procedure
• Post findings (inline comments + summary)

Scope reviewed: all 4 changed files (cant-fail-scan.test.sh, README.md, CHANGELOG.md, and the new .changes/testing/ fragment).

Findings: none.

What I checked:

• The new assertion (cant-fail-scan.test.sh line 2052) checks that rule-throw-only-oracle's remedy does not contain the rule-weak-oracle remedy string. I read cant-fail-scan.sh: weak-oracle:* emits "Assert the value the code should produce…" (L1083) and throw-only-oracle:* emits a distinct string, "Assert what the constructed value should hold…" (L1104). The two never overlap, so the new assert_not_contains is correct and matches the existing pattern used for the same purpose at L2045-2047 (weak-oracle vs. zero-assertion).
• README.md's two count edits (7→8 of 13) are consistent with each other and with the row's prose; the untouched "seven other report-only rules" phrase in the Confidence sentence refers to a different count (report-only rules excluding rule-inert-assertion, which is already in the high-confidence group) and correctly was not touched.
• CHANGELOG.md's new [3.8.1] entry correctly increments from the current [3.8.0] on origin/main. Note: the PR body says this is a [3.7.1] entry, but the actual diff adds [3.8.1] — a stale description, not a defect in the change itself (not flagged as a finding; just worth knowing if you re-read the PR body later).
• The new .changes/testing/ fragment uses bump: none correctly; testing is listed in scripts/fragment-plugins.txt, matching the PR's "no plugin changelog fragment" claim.
• I could not execute the test suite in this sandbox (Bash commands here require interactive approval I can't grant), so the "assertion passes" conclusion above is from static source reading, not an independent run — the author's own run (1,283 PASS / 0 FAIL, stopped partway) and CI's test-bash lane remain the executed signal.

claude-lane-summary: lane=claude-review sha=4182a80dd2d98385524d06c8fa213815e660dc39 run=38403720718

[github-actions]

claude-security-review has reviewed this pull request through 4182a80; a later push is reviewed from there.

[github-actions]

claude-review has reviewed this pull request through 4182a80; a later push is reviewed from there.