test(descriptions): no advertised tool description carries leaked indentation - #380
Conversation
…entation The narrow, sound half of issue 378. Seven caller-facing messages were found carrying runs of leaked indentation — a Rust string continuation collapsed onto one source line keeps its whitespace — and all seven are fixed in #376/#377. This is the gate that stops the same thing recurring on the one surface where the population is complete. ## Why descriptions and not "messages" #378 records that there is no cheap sound gate over "every string that can reach a caller": * A SOURCE scan cannot distinguish a collapsed continuation from a correct one without exclusion rules for whitespace test fixtures, embedded ObjectScript whose indentation is deliberate, and aligned trailing comments. Measured: the naive version reported 592 hits across 68 files, every one a false positive, because it matched across lines and so read the source layout rather than the runtime string. * Message CONSTANTS are a sound population but a tiny one — three in this crate — and would have caught none of the seven, which were literals inside functions. * Messages built at call time are only reachable by triggering their paths. Descriptions are the one population that is complete, wire-facing and free: every one goes to every client on every tools/list, and advertised_tools() returns all of them with no IRIS connection. So this asserts what it can assert exhaustively and says plainly that it is not the whole problem. ## What it checks, and its controls All four toolsets, because a description can be advertised by one and not another. Two controls guard against a vacuous pass: the tool list must hold at least 20 entries, and at least 20 must actually carry a description — a zero from an empty list would otherwise read as clean. The detector has its own test, using the real text as it came back from the tool before the fix. That matters here specifically: the first version of this scan produced 592 false positives, and a version tuned the other way would have produced zero real ones with nothing to show the difference. ## Verification fmt, clippy -D warnings, cargo build --workspace, and the gate with IRIS env unset: 5 result lines, 0 FAILED, both tests confirmed by name, a fabricated name returning 0. Two mutations, each red and restored: * a collapsed continuation injected into iris_execute's real description — caught, naming the tool and reporting a 22-space run. * the tool list truncated to empty — caught by the control, "advertised only 0 tools, so a clean result would prove nothing", rather than passing as clean. Currently zero descriptions are affected, so this commit is green on the tree as it stands: it locks in a property rather than fixing one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ran this PR's test-after-put against a real instance for the first time. The behaviour is right in all five cases — a green suite gives test_ok: true; a RED suite gives test_ok: false while the write's own success stays true, which is the #310 separation working live; no compile gives test_skipped; a compile failure gives COMPILE_ERROR with the write envelope untouched and no test block; and no `test` parameter adds nothing at all. What was wrong is the text. The skip message came back as: not run: `test` needs a COMPILED class and this call did not compile. Pass compile=true Four messages, all in this PR: the three TestGate::skipped_reason arms and the unparseable-result note. Same collapsed line-continuation as the seven fixed in #376/#377 — I fixed those on the branches I happened to be working in and did not check the others. ## The scan that found the rest, and the one that did not Running the earlier detector over every branch was useless: master-descended branches report 15-40 hits each, because most matches are whitespace test fixtures, embedded ObjectScript whose indentation is deliberate, and aligned doc-comment tables. No conclusion is available from those numbers. Scoping it to the lines a branch ADDS (`git diff origin/master...origin/<br>`, `^+` only) makes the population precise. Across all fifteen open PR branches that gives four real leaks, all four here. Everything else the scan flags on those branches is intentional: measurement tables in doc comments, the deliberate `"ERROR #5002: "` truncation fixture, the real IRIS SQLCODE text in #368's fixtures (which genuinely contains double spaces), and the leaked string the #380 detector test asserts on. Verified at runtime after the fix by driving the built binary: the skip message contains no double space. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Recording the #378 linkage — nothing here references it, so merging would leave it open — but not adding Measured against the diff: #378 is about "caller-facing messages" — its reported instance was a The remaining half, and why it is blockedI scoped it on the issue earlier: reconstruct each string literal as the compiler would, then flag only internal multi-space runs — which structurally excludes whitespace fixtures (trim-empty), deliberate ObjectScript indentation (line-leading) and So the gate is variant D plus a justified seven-entry allowlist (verbatim PSQL/IRIS text, deliberate column alignment, embedded ObjectScript in a regular string). It cannot land before #377. The single true positive in variant D is So #378 needs this PR and a follow-up gate after #377. Closing it is a merge-time call. |
Ran this PR's test-after-put against a real instance for the first time. The behaviour is right in all five cases — a green suite gives test_ok: true; a RED suite gives test_ok: false while the write's own success stays true, which is the #310 separation working live; no compile gives test_skipped; a compile failure gives COMPILE_ERROR with the write envelope untouched and no test block; and no `test` parameter adds nothing at all. What was wrong is the text. The skip message came back as: not run: `test` needs a COMPILED class and this call did not compile. Pass compile=true Four messages, all in this PR: the three TestGate::skipped_reason arms and the unparseable-result note. Same collapsed line-continuation as the seven fixed in #376/#377 — I fixed those on the branches I happened to be working in and did not check the others. ## The scan that found the rest, and the one that did not Running the earlier detector over every branch was useless: master-descended branches report 15-40 hits each, because most matches are whitespace test fixtures, embedded ObjectScript whose indentation is deliberate, and aligned doc-comment tables. No conclusion is available from those numbers. Scoping it to the lines a branch ADDS (`git diff origin/master...origin/<br>`, `^+` only) makes the population precise. Across all fifteen open PR branches that gives four real leaks, all four here. Everything else the scan flags on those branches is intentional: measurement tables in doc comments, the deliberate `"ERROR #5002: "` truncation fixture, the real IRIS SQLCODE text in #368's fixtures (which genuinely contains double spaces), and the leaked string the #380 detector test asserts on. Verified at runtime after the fix by driving the built binary: the skip message contains no double space. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ether anything compiled (#371) * feat(iris_doc): run the named test after a compiled write, and say whether anything compiled Items 2 and 4 of issue 327, measured over 31,009 parsed `tool_use` blocks from 364 transcripts. `iris_doc(put)` is immediately followed by `iris_test` 579 times. A boolean would cover only the 346 where the class written IS the class tested; a STRING covers all 579, because in the other 233 the suite lives beside the class (`Hospital.BO.PatientDb` -> `Hospital.Tests.BO.PatientDbTest`). The decision is a pure `TestGate` with four states, not a boolean, because the ways a test does NOT run need different fixes from the caller: * `Run(pattern)` — the payload reports a compiled class. * `NotRequested` — nothing added to the payload at all. A `test_skipped` on every ordinary put would be noise on the 2,979 measured puts that want none of this. * `SkippedCallFailed` — the write did not succeed. Its own envelope is returned UNTOUCHED: it already carries the compile error, SCM elicitation or refusal the caller must act on, and rebuilding it to add a note would risk dropping the isError flag and hints that are the answer. * `SkippedNotCompiled { compile_requested }` — two different sentences, because "you did not ask for a compile" and "the compile produced nothing" are different states and need different fixes. Keyed off the payload's `compiled`, not off the mode, so it needs no list of modes to keep in step with `DocMode`. The write's verdict and the test's verdict stay separate (#310): `success` stays the WRITE's — the document landed — and the test arrives as `test` with `test_ok` beside it. A red test must not read as a failed write, and a failed write must not read as a red test. Sequenced in the `iris_doc` tool method rather than inside `doc::handle_iris_doc`, because running a suite is `iris_test`'s own 600-line job (polling, the result query, the per-case shaping of #233/#273). Calling the tool means the two paths cannot disagree about what a red test looks like. 89 of the measured `put` -> `iris_compile` pairs are a put that did not pass `compile`, followed by a compile of the SAME class. Issue 327 records that as a 3% tail and proposes the cheap option rather than flipping the default: have the put say so. The no-compile branch carried NO `compiled` key at all, so "was it compiled?" was answerable only by noticing an absence — and an absence reads as "does not apply here", not as "no". All three write outcomes now report `compiled` AND `compile_requested`, so the states are distinct from those two fields alone: (false,false) written and nobody asked, (false,true) the compile ran and failed, (true,true) compiled clean. fmt, clippy `-D warnings`, `cargo build --workspace`, and the gate with IRIS env unset. 15 new tests. 10 mutations applied one at a time — including "a blank test counts as a request", "a red test rewrites the write's own verdict", "iris_doc stops handing its result to the sequencer" and "the sequencer stops calling iris_test" — each red on its named tests, restored green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(iris_doc): four TestGate messages carried leaked indentation Ran this PR's test-after-put against a real instance for the first time. The behaviour is right in all five cases — a green suite gives test_ok: true; a RED suite gives test_ok: false while the write's own success stays true, which is the #310 separation working live; no compile gives test_skipped; a compile failure gives COMPILE_ERROR with the write envelope untouched and no test block; and no `test` parameter adds nothing at all. What was wrong is the text. The skip message came back as: not run: `test` needs a COMPILED class and this call did not compile. Pass compile=true Four messages, all in this PR: the three TestGate::skipped_reason arms and the unparseable-result note. Same collapsed line-continuation as the seven fixed in #376/#377 — I fixed those on the branches I happened to be working in and did not check the others. ## The scan that found the rest, and the one that did not Running the earlier detector over every branch was useless: master-descended branches report 15-40 hits each, because most matches are whitespace test fixtures, embedded ObjectScript whose indentation is deliberate, and aligned doc-comment tables. No conclusion is available from those numbers. Scoping it to the lines a branch ADDS (`git diff origin/master...origin/<br>`, `^+` only) makes the population precise. Across all fifteen open PR branches that gives four real leaks, all four here. Everything else the scan flags on those branches is intentional: measurement tables in doc comments, the deliberate `"ERROR #5002: "` truncation fixture, the real IRIS SQLCODE text in #368's fixtures (which genuinely contains double spaces), and the leaked string the #380 detector test asserts on. Verified at runtime after the fix by driving the built binary: the skip message contains no double space. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The narrow, sound half of #378. Seven caller-facing messages were carrying runs of leaked
indentation — a Rust string continuation collapsed onto one source line keeps its whitespace — and all
seven are fixed in #376/#377. This is the gate that stops it recurring on the one surface whose
population is complete.
Why descriptions, and not "messages"
#378 records that there is no cheap sound gate over every string that can reach a caller:
["", " "]), embedded ObjectScript whose indentation is deliberate (" Do ..RunUser()"), and aligned comments. Measured: 592 hits across 68 files, all false positives — it matched across lines, so it read the source layout rather than the runtime stringDescriptions are the one population that is complete, wire-facing and free: every one goes to every
client on every
tools/list, andadvertised_tools()returns all of them with no IRIS connection. Sothis asserts the surface it can assert exhaustively, and the doc comment says plainly that it is not
the whole problem.
Controls, because a clean result here is the easy thing to get wrong
fix, plus the repaired form and a newline case that must not be flagged.
That last one matters here specifically: the first version of this scan produced 592 false positives,
and a version tuned the other way would have produced zero real ones with nothing to distinguish the
two.
Verification
cargo fmt --all -- --checkclean,cargo clippy --workspace --all-targets -- -D warningsclean,cargo build --workspace, and the gate with IRIS env unset: 5 result lines, 0 FAILED, both testsconfirmed by name, a fabricated name returning 0.
2 mutations, each red and restored:
iris_execute's real descriptionZero descriptions are affected today, so this is green on the tree as it stands: it locks in a
property rather than fixing one. #378 stays open for the source-scan scoping decision.