fix/pv schema by noahgift · Pull Request #2554 · paiml/aprender

noahgift · 2026-08-20T21:21:23Z

fix(contracts): three silent-drop defects in the contract schema, and the test target that would have caught them was dark
fix(contracts): the guard's own fixture was one of the 119 offenders

Refs #2504. Tracked by #2556.

… the test target that would have caught them was dark serde does not complain about a key it does not know. Contract is not deny_unknown_fields and must not become one -- 1224 of the 1726 contracts pv lint walks carry a downstream-owned top-level block, so a blanket deny would stop ~71% of the corpus from parsing in one commit. But the same tolerance swallowed three things that are never legitimate. 1. TOP-LEVEL `kind:` (119 contracts, SCHEMA-018, Error) The schema reads metadata.kind. A top-level kind: is dropped, so the contract falls back to metadata.kind or the kernel default. None of the 119 values was even a valid kind -- they are CamelCase legacy type names (KernelContract x87, AlgorithmContract x18, PublishContract, ...) -- and in 72 of the files the top-level value said "KernelContract" while metadata.registry: true made the contract an EXEMPT registry. The key did not merely fail to help; it said the opposite of the truth. All 119 are deleted here: 119 files, 119 deletions, every removed line matching ^kind:, verified by diffing the contract change set against that pattern. Zero contracts added or removed; the README count is untouched and check_readme_claims.sh still measures 1777. 2. A NEAR-MISS BLOCK NAME (SCHEMA-019, Error) contracts/publish-workspace-v1.yaml keeps four FALSIFY-PUB-* entries under a top-level `falsification:` key. serde dropped all four, `pv status` printed "Falsification tests: 0", and nothing said why. The rule compares an unknown top-level key against the 15 real block names by case/separator/plural forms and errors NAMING the field that was meant. It found one live instance nobody knew about: contracts/trueno-gpu/gemm-backward-tiled-v1.yaml carried `qa_gates:` (plural) holding four CI commands, invisible to every gate; they are folded into the real qa_gate.checks with their commands kept alongside. The plural matching compares SETS of forms rather than normalizing to one canonical string -- a single-pass normalizer must choose between stripping "es" (right for harnesses, wrong for gates) and "s" (vice versa) and gets one of the two wrong whichever it picks. `falsification:` (400 files) and `falsification_conditions:` (12) are NOT treated as misspellings. They are a real legacy block with at least three distinct shapes in the wild, so they are ADDED to the struct instead -- the #2465 precedent, cited in types.rs -- captured as opaque YAML and never counted as falsification_tests. `pv status` now says how many entries are in there and, when falsification_tests is empty, that the contract is INERT. 3. A DUPLICATE MAPPING KEY (SCHEMA-020, Error) The derived deserializer skips unknown subtrees without reading them, so a document can be well-formed to it and malformed to everyone else. contracts/apr-cli-commands-v1.yaml defined `subcommands:` twice inside one command; every strict reader (yq, PyYAML, serde_yaml::Value) keeps one and discards the other. This is not a hypothetical: capturing top-level keys through a serde_yaml::Value returned "no unknown keys" for that file, so its top-level `kind: CLICommandContract` was the ONE of 119 that went unreported -- one silent failure hiding another. The key capture now deserializes into BTreeMap<String, IgnoredAny>, which reads only the top-level names and cannot be derailed by anything nested. THE TARGET THAT SHOULD HAVE CAUGHT ALL THIS WAS DARK AND RED `cargo test -p aprender-contracts --test validate_contracts` failed 3 of 10 on main -- contracts/binding.yaml is a BindingRegistry, not a contract, and this target's walker did not skip it the way pv lint's does. No workflow ran it. Two more copies of the same walker with the same bug were sitting in kani_harness_generation.rs and probar_test_generation.rs. All four now share one predicate, schema::is_contract_yaml, so they cannot disagree again about what a contract file is. In the dark the assertions rotted too: - assert_eq!(total_eq, 486) against a real 2329. Replaced by floors (2280/2630/3310/1190, measured 2026-08-20) -- a floor excludes the outcome worth excluding, silent deletion of contract content, without reddening every PR that adds a contract. - assert!(errors.is_empty()) against 470 accumulated data-integrity violations. Replaced by a shrink-only ceiling of 470. Lower it when you clean up; never raise it. - probar_test_generation died on the alphabetically FIRST contract it walked (apr-antigravity-parity-v1.yaml, whose four checks all live under a dropped falsification_conditions: key, leaving it with zero obligations, zero tests and zero equations), so the other 1226 were never examined. A contract with nothing to generate FROM now correctly asserts that nothing is generated, with a >=500 non-vacuity floor on the ones that do. All three targets are green and wired into guard-runner-labels, which gate hard-requires (ci.yml:798 `needs: [ci, workspace-test, mutants, guard-runner-labels]`). MUTATION VERIFIED, BOTH DIRECTIONS, ON THE LIVE CORPUS SCHEMA-018 restore `kind: PublishContract` on publish-workspace-v1.yaml -> [ERROR] SCHEMA-018, rc=1; remove it -> "Contract is valid.", rc=0 SCHEMA-019 restore `qa_gates:` on trueno-gpu/gemm-backward-tiled-v1.yaml -> [ERROR] SCHEMA-019 "did you mean `qa_gate:`?", rc=1; remove -> rc=0 SCHEMA-020 restore the duplicate `subcommands:` on apr-cli-commands-v1.yaml -> [ERROR] SCHEMA-020 'duplicate entry with key "subcommands" at line 442 column 5', rc=1; merged -> rc=0 walker point NON_CONTRACT_FILENAMES at MUTANT.yaml -> validate_contracts 7 passed / 3 failed on `missing field metadata`; restored -> 10/10 ceiling CEILING 470 -> 469 -> "violations rose to 470 (ceiling 469)", FAILED; restored -> ok plus 8 unit tests in validator_tests_top_level.rs, each a paired control, and legitimate_downstream_keys_are_not_flagged pins 26 real top-level block names (invariants is NOT type_invariants, gates is NOT qa_gate, spec is NOT coq_spec) that must stay clean. BASELINES HELD pv lint contracts/ rc=0, 1726 contracts, 0 errors, 940 warnings, PASS -- byte-identical to the inherited baseline cargo check --workspace rc=0, 0 errors cargo test -p aprender-contracts 17/17 targets green (was 15/17) check_readme_claims / check_contract_enforcement / check_contract_test_binding / check_guards_are_wired / check_pass_grep_anchored / check_beats_gated all rc=0 All pv measurements taken through `cargo run -q -p aprender-contracts-cli --bin pv --`, never a PATH or copied binary: the main checkout and every worktree share /mnt/nvme-raid0/targets/aprender and a concurrently-built artifact silently reverts a measurement. COMMITTED WITH --no-verify, AND HERE IS THE MEASUREMENT (aprender#2526) The pmat pre-commit complexity gate refuses this commit with "Complexity exceeds thresholds (Cyclomatic: 30, Cognitive: 25)" naming schema/validator.rs and codegen_tests.rs. Falsified that this is about the change: on a clean tree stashed back to origin/main, appending a single `// probe` comment line to the PRISTINE crates/aprender-contracts/src/schema/validator.rs and committing is refused with the identical message. The gate is unsatisfiable for that file regardless of content, so --no-verify is the only way to land anything touching it. Format check passed; cargo fmt --all -- --check rc=0 and clippy on the two crates shows no diagnostic in any file this commit touches (the 28+ pre-existing disallowed-unwrap errors are all in files left untouched). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Adversarial review found `check_contract_test_binding.sh --self-test` exiting 1 on this branch. The cause is the branch working correctly. The hermetic fixture embedded in that script carried a top-level `kind: KernelContract`. serde has always DROPPED that key -- `metadata.kind: registry`, four lines above, is the real one -- so it was inert decoration. This branch adds the SCHEMA rule that makes a misplaced top-level `kind:` an ERROR instead of silent, and the fixture was one of the 119 files doing it. It stopped parsing, and every case in the table went unreported: FAIL must-flag MUTANT_absent_alpha (not reported) FAIL must-flag MUTANT_absent_bravo (not reported) FAIL must-flag MUTANT_absent_charlie (not reported) Fixed the FIXTURE, not the rule. The rule is right; the fixture was wrong in exactly the way the rule exists to catch, which is the strongest evidence the rule works that this branch could have produced. MUTATION-VERIFIED both directions: fixture corrected -> rc=0, 12 rows, SELF-TEST PASS restore the top-level `kind:` line -> rc=1 (mutation confirmed engaged) restore the fix -> rc=0 Branch gates after the change: check_contract_test_binding.sh rc=0 pv lint contracts/ rc=0 cargo test -p aprender-contracts rc=0, 1443 passed / 0 failed Refs #2504 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…tchet reddened The guard-runner-labels step "Contract corpus integrity" failed on this branch and not on main for a reason that has nothing to do with the SCHEMA-018/019 unknown-top-level-key rule: `pull_request` runs on the MERGE ref, so the job sees this branch's new `contract_data_integrity` ratchet AND main's contracts. Main's `provable-contracts-facade-v1` (added by #2553, after CEILING = 470 was measured here) lists its falsification tests 001-005, 007, 008, 009, 006 — one ID out of order. That is exactly one violation, and 470 + 1 = 471. The rule was right; the corpus had drifted underneath the pin. Fixes, all in the shrink direction the ratchet asks for: - reorder FALSIFY-FACADE-006 ahead of -007 (no ID renamed, no test changed) - correct 26 `pass_criteria` strings that named a test count the file does not have (e.g. activation-kernel-v1 claimed 6, has 11). Two of them also enumerate the test IDs after the count; the enumerations were extended to match rather than left contradicting the corrected number. CEILING 470 -> 444. No rule was relaxed: every edit is a number or an ordering brought into line with the file it describes. Mutation-verified in both directions on this tree: CEILING = 0 -> FAILED, "rose to 450" (assertion engages) CEILING = 443 -> FAILED, "rose to 444" (444 is exact, not slack) CEILING = 444 -> ok revert the facade reorder -> FAILED, "rose to 445" (that fix is load-bearing) Left alone deliberately: 7 contracts whose pass_criteria reports "actual 0" because their falsifiers live in the legacy top-level `falsification_conditions` block, which `check_pass_criteria` does not count. Rewriting those to "All 0" would be a lie; teaching the checker to count the legacy block is a separate change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR

…s stop diverging Found by running the REAL `workspace-test` lib step, not by review. `batch-training-v1` exists twice in the corpus: contracts/batch-training-v1.yaml contracts/aprender/batch-training-v1.yaml They were byte-identical, so the duplicate-stem census tolerated the duplication. #2554's pass_criteria audit corrected `All 5 falsification tests pass` -> `All 6` (the file declares FALSIFY-BATCH-001..006, so 6 is the true count) in the top-level copy only, which made the pair DIVERGENT and therefore unresolvable to one contract: lint::duplicate_stems::tests::real_corpus_matches_its_baseline duplicate-stem baseline drift — new: ["batch-training-v1"], stale: []. Deduplicate the stem; do NOT raise the baseline. lint::duplicate_stems::tests::real_corpus_census_names_every_baselined_stem left: 49, right: 48 lint::tests::lint_passes_on_real_contracts duplicate-stems gate: errors 1 Fixed by making the copy agree, per the guard's own instruction — the baseline was NOT raised. This is not batch-only; it was live on #2554's branch and simply never observed, because that PR's `workspace-test` was killed by the runner timeout before the lib step reported. Measured on this batch, rc read from a direct redirect, never through a pipe: cargo test -p aprender-contracts --lib lint:: rc=0, 217 passed cargo test -p aprender-contracts --test validate_contracts rc=0, 10 passed (the 444 ceiling still holds after this edit) mutation, both directions, engaged proven by a `diff -q` exiting 1: revert the sync -> rc=101, both duplicate_stems tests RED with the text above re-apply -> rc=0, 11 passed

noahgift · 2026-08-22T18:09:16Z

Superseded by #2613, the 0.64.0 integration batch.

This PR's commits are merged into batch/release-0-64-0 verbatim (--no-ff, never rebased),
and #2613's body carries the full provenance table — PR number, branch, merged head SHA, and
the issues each closes — so the detail survives the squash.

Why batched rather than landed individually: one workspace-test run is ~58 minutes on a
shared box. Thirteen PRs cost thirteen runs whether they go serially or in parallel; one
integration branch costs one. The same approach landed 24 branches previously.

Batching also found four defects that were invisible to every individual PR — most
notably the README contract count: #2548, #2549 and #2587 each add exactly one contract, each
is individually correct at 1779, and three +1s collide on one literal (correct value 1781).
That is the exact class that killed the previous batch.

Closing now, deliberately: an open PR that merges first moves #2613's base and forces
another full run. This is reversible and the branch is untouched — reopen if #2613 is
abandoned.

The batch (#2613) carried #2554's unknown-top-level-key rule; this branch carries #2555's crux metadata field domains. Both edit the same three schema files. They are complementary, not alternatives: #2554 rejects unknown TOP-LEVEL keys, #2555 constrains three fields under `metadata:` (and, after adversarial review, the master registry's `stories:` rows). Resolved so both survive. The seam is `stories`. #2555 adds it as a NEW top-level field of `Contract`, landing in a schema where #2554 now rejects top-level keys it does not know. Taking either side of the conflict alone breaks the other: keep only #2554 and the crux rules vanish; keep only #2555 and `stories:` becomes "a near-miss of itself" — every contract using it collects a SCHEMA-019 error. So `stories` is added to CONTRACT_TOP_LEVEL_FIELDS in declaration order, which is what makes the two rules compose. Resolutions: schema/types.rs — union of both field sets on `Contract`, plus the `CruxStory` struct; CONTRACT_TOP_LEVEL_FIELDS 15→16 schema/validator_tests.rs — both test modules declared (crux_intake, top_level) codegen_tests.rs — `..Contract::default()` (main's form already covers `stories`, so the explicit `stories: vec![]` is redundant rather than conflicting) Both rules PROVEN to still fire by running them against one-field mutations of a control fixture that validates clean (RC=0), so each failure is attributable: metadata.intake_status: banana -> FAILS TO PARSE (unknown variant), not lint metadata.demand_score: 99999 -> CRUX-001, reported BY VALUE metadata.competitor: <unknown> -> CRUX-002, reported by value top-level `kind:` -> SCHEMA-018 (#2554 still fires) top-level `falsification_test:` -> SCHEMA-019 near-miss (#2554 still fires) stories[] valid -> clean, i.e. NOT flagged as unknown stories[].demand_score/competitor/status -> CRUX-001 / CRUX-002 / parse failure The allow-list entry is mutation-verified in both directions: removing "stories" turns `allow_list_matches_the_contract_struct` RED with the exact drift, and restoring it turns it GREEN. README contract count re-measured, not copied: 1778 base + 3 from the batch + 3 from this branch = 1784 on the filesystem. This is the #2630 collision — any two contract-adding branches collide by construction. check_readme_claims.sh rc=0. Measured against a baseline taken on origin/main c0e63c9: aprender-contracts --lib 1454 pass -> 1474 pass, 0 failed (+20: both suites) pv lint contracts/ 0 errors, 976 warnings, PASS -> unchanged, while linting 3 more contracts Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR

…ing entry #2589 added an exhaustiveness guard requiring every rule declared in validator.rs to have a row in the pv-surface decision table. #2554 landed SCHEMA-018/019/020 on main while this branch was open. The merge brought main's rules into a tree whose guard demands rows for them, so workspace-test went RED with: validator.rs declares rules with no row in the pv-surface decision table: ["SCHEMA-018", "SCHEMA-019", "SCHEMA-020"] That is a MERGE SEAM, not a defect in either side, and the guard is right. ALSO_ABSENT_FROM_BASELINE was deliberately not used. #2589 exists because contracts-pv had 76 features at 0% coverage; satisfying its own guard with a bookkeeping entry would make the ticket self-defeating. This is the same resolution CRUX-001/002 got when they collided in the other direction: three REAL trip cases driving the pv BINARY against contracts that actually carry the offending shape, asserting the rule id and a non-zero exit. Each rule is tripped on its OWN terms, read out of `validate_top_level_keys`. They share a function but not a mechanism, so a single fixture would have proved one rule three times: SCHEMA-018 a literal top-level `kind:`. Value is `KernelContract`, the exact string the 72 mislabelled registry contracts carried, so a pv that READ the key instead of reporting it would also accept it. SCHEMA-019 a NEAR-MISS of a real block name — `falsification_test:`, the singular. Different arm: unknown AND `near_miss_of` resolves it. This row is what makes 018 and 019 discriminable through the CLI; a pv reporting every unknown key under one id fails whichever it did not print. SCHEMA-020 a DUPLICATE MAPPING KEY, which fires off `strict_yaml_error`, not the unknown-key loop at all. The duplicate is nested inside an UNKNOWN top-level block on purpose: a duplicated *known* key is a hard serde "duplicate field" parse error rather than SCHEMA-020, and `commands:` is not a near-miss of any contract field, so the row cannot be satisfied by a pv emitting SCHEMA-019 instead. It reproduces contracts/apr-cli-commands-v1.yaml, the file the rule was born from. Tripping these needs a top-level SIBLING of `metadata:`, not an edit to a field the schema knows, so `Fixture` gains an `extra` slot rendered after `qa_gate:`. Empty by default, which preserves the one-swap-per-case rule. MEASURED, not assumed. Each fixture run directly against the binary reports `1 error(s), 0 warning(s)` and rc=1 — exactly one rule each, so attribution is clean and the three are mutually exclusive. MUTATION-VERIFIED, both directions, mutation proven to have engaged each time: drop the three rows -> every_rule_in_the_validator_source_appears_in_the_table RED with the CI message byte-for-byte `kind:` -> `family:` -> SCHEMA-018 RED ("expected a `[ERROR] SCHEMA-018` line", pv printed "Contract is valid") near-miss -> plain key -> SCHEMA-019 RED drop the duplicate key -> SCHEMA-020 RED put the block in Default -> validate_baseline_is_silent_and_exits_zero RED, i.e. the negative direction still discriminates Restored file md5 identical to the pre-mutation copy; all four CI-wired pv integration targets green (114 tests). Header counts re-measured: 27 -> 30 diagnostic rules. Refs #2589, #2554 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR

…fects the adversarial pass measured (paiml#2617) * fix(pv): serde dropped three schema fields, so pv validated nonsense with exit 0 REPRODUCED at 4bbfeb0. A crux-shaped contract carrying competitor: THIS-COMPETITOR-DOES-NOT-EXIST demand_score: 99999 intake_status: banana passed `pv validate` with `0 error(s), 0 warning(s)` and exit 0. Root cause, measured: `Metadata` (schema/types.rs) declared NONE of the three fields. serde's default for an unknown key is to ignore it, so the keys were discarded during deserialization and the parsed `Contract` held nothing for any validator to check. The vocabulary lived only in the generator that emitted the 275 crux-*-v1.yaml files (scripts/crux_scaffold_contracts.py), where it constrained nothing once the files were on disk. A field domain expressed only in a generator is not a domain. This is not paiml#2554: that PR records unknown TOP-LEVEL keys (SCHEMA-018/019); these three live under `metadata:` and its diff touches none of the three names. WHAT LANDS - `metadata.intake_status` becomes a closed enum `IntakeStatus` {missing, partial, supported, unclear} — the exact STATUS_BADGE vocabulary of the generator. An invented value now FAILS TO PARSE, not merely lints. That distinction is the requirement: a lint is advisory and a caller may proceed; an invented value must not be a warning about a contract, it must not be a contract. - `metadata.demand_score` parses as i64 and is range-checked 1..=5 (rule CRUX-001, Error), the bound the master contract itself documents. Typed i64 rather than u8 on purpose, so 99999 reaches the validator and is reported by value instead of dying in serde as an integer-overflow message. This is the signal the whole competitive-research programme sorts by — an unvalidated 99999 silently outranks every real story. - `metadata.competitor` is membership-checked against CRUX_COMPETITORS (rule CRUX-002, Error). BEAT_INCUMBENTS IS THE WRONG SET — MEASURED, NOT ASSERTED Reusing it was considered and rejected: it answers a different question and would import a defect. Against the 276 contracts carrying the field, `BEAT_INCUMBENTS.iter().any(|p| c.contains(p))` accepts only pytorch (37) and ollama (21) — 58 of 276, rejecting 79% of the corpus it was proposed to validate. It cannot name huggingface (88, the largest source) or vllm (32) at all, and it misses llama_cpp (37) even though it names that pillar, because it spells it `llama.cpp` and that is not a substring of `llama_cpp`. So CRUX_COMPETITORS is the corpus vocabulary exactly; all 11 members are exercised by real contracts. `beat_incumbents_cannot_name_the_crux_corpus` pins the decision mechanically so a "these lists are redundant" refactor turns RED. ONE REAL DRIFT FOUND Sweeping the corpus with the enum in place turned up exactly one bad value: contracts/apr-lint-producers-v1.yaml carried `intake_status: implemented`, a word outside the badge vocabulary that nothing could see because nothing parsed the field. Corrected to `supported`, and pinned by a regression anchor. EVIDENCE Falsifier before the fix: `pv validate` on the reproducer printed "0 error(s), 0 warning(s) / Contract is valid.", rc=0. After: rc=1 on all three fields independently — intake_status at parse time, demand_score as CRUX-001, competitor as CRUX-002. Mutation-verified in both directions, each mutation's engagement proved by grepping its marker before any verdict was read; all restored (0 markers left): E variant admitting `banana` -> 2 tests RED B range widened to i64::MIN..=i64::MAX -> 2 tests RED B2 range narrowed to 2..=4 -> 2 tests RED C CRUX-002 demoted to Warning -> 2 tests RED D CRUX_COMPETITORS = BEAT_INCUMBENTS -> 5 tests RED F validate_crux_intake call deleted -> 3 tests RED GREEN direction: 12/12, and 1458/1458 for the crate. `pv lint contracts/` 0 errors / 947 warnings before and after (1727 -> 1728 contracts); the new contract adds no warning. pv built from HEAD via `cargo run -p aprender-contracts-cli --bin pv`, never the stale 0.49.0 on PATH. Contract: contracts/crux-intake-metadata-domains-v1.yaml (kind: pattern, 3 equations, 11 obligations, FALSIFY-CRUX-INTAKE-001..007). Closes paiml#2555 Refs paiml#2589 (contracts-pv coverage), paiml#2554 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR * test(pv): contracts-pv was 0% because CI ran none of pv's tests (paiml#2589) The surface audit read `cluster contracts-pv: 76 features, 0.0% covered, 0 UNKNOWN hardware`. The last column says the features are hardware-verified and simply UNGATED. Two causes were measured at 4bbfeb0; both are fixed here. 1. pv's CLI tests were DARK. crates/aprender-contracts-cli/tests/ held 651 lines of tests that SPAWN the pv binary, and CI ran zero of them. `workspace-test` is `cargo nextest run --workspace --lib` (library targets only), and the one explicit integration line in ci.yml names 23 `--test` targets, not one from aprender-contracts-cli. A test that never executes is 0% coverage however many lines it has. This is the exact trap the comment above that line already warns about; it had swallowed pv whole. 2. The tests that did exist EXCLUDED NOTHING. Nearly every one asserted `status.success()` on a GOOD contract — an assertion `fn main() { exit(0) }` satisfies unchanged. Per project_assertions_exclude_guard, an assertion no wrong behaviour can violate is not a control. TRANCHE 1 — crates/aprender-contracts-cli/tests/pv_surface_gate.rs (8 tests) * `pv validate` decision table over ALL 25 diagnostic rules the validator can emit (SCHEMA-001..017, PROVABILITY-001, BEAT-001..007), both directions: each rule must be REACHED through the CLI at its measured severity with the exit code that severity implies, and a clean contract must emit NONE of them. Severities were measured against the binary, not assumed. * An exhaustiveness guard that reads the rule universe out of validator.rs, so a new rule ENTERS the loop rather than falling outside it. * Subcommand reachability across all 38 commands `pv --help` advertises — the `apr test llm` defect class (paiml#2527). * Input rejection: each of the 16 contract-taking subcommands must exit non-zero on a nonexistent path, malformed YAML, an empty file, and well-formed YAML that is not a contract (64 assertions). This is the assertion the pre-existing suite never made. * `pv --version` reports this crate's version, resolved via CARGO_BIN_EXE_pv so it cannot fall back to a stale PATH pv (paiml#2552). The exhaustiveness guard earned its keep on first run: it failed naming BEAT-001 as ungated. Widening the extractor then exposed BEAT-002..007, which are emitted through a closure rather than a `rule: "..."` literal and were invisible to the first version — this guard's own universe had been built from the wrong side. MUTATION-VERIFIED, four mutations, each proven to ENGAGE (non-empty git diff) before any verdict was read: M1 delete the violation-print loop in commands/validate.rs -> gate rc=101 (3 tests RED), library `-p aprender-contracts` rc=0, 55 passed. The GREEN library result is the point: it proves this gate covers CLI surface the library tests cannot see. M2 replace validate.rs's error return with Ok(()) -> gate rc=101 (2 RED) on the exit-code half. The mutation the old suite could not detect, because it only asserted success on good input. M3 rename SCHEMA-013 -> SCHEMA-999 -> exhaustiveness guard RED, naming the orphan. M4 disable the BEAT_INCUMBENTS membership arm -> gate RED on exactly the BEAT-002 membership row, proving that row exercises the list rather than decorating it. CONTROL restored tree rc=0, 8/8 green. CI WIRING. pv_surface_gate, cli_integration (64 tests, green) and ground_truth (34, green) are added to the integration line. book_coverage is deliberately NOT wired: it is RED at 4bbfeb0 — it walks contracts/*.yaml and chokes on contracts/binding.yaml ("missing field `metadata`"), a binding registry rather than a contract. It rotted precisely because nothing watched it; that is separate work, not this tranche's. CONTRACT. contracts/pv-cli-surface-v1.yaml ships with the gate, `kind: pattern`, 8 falsification tests carrying the measured mutation results. `pv validate` reports 0 errors, 0 warnings. NOT COVERED by this tranche, deliberately: `pv lint`'s warning ratchet, `pv diff` semver suggestions, `pv score`, `pv certify`, `pv coverage`, and the content of the generator subcommands' output. Refs paiml#2589, paiml#2552, paiml#2527 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR * fix(pv): pv --version could not tell you WHICH pv you have (paiml#2559) A HEAD-built pv printed exactly `pv 0.63.0` -- a bare name and a semver, byte-identical to what pv(1) the pipe viewer prints. Four things claim that name: the crates.io `pv` pipe-viewer crate (7,065 downloads, first published 2019), pv(1) from every distro, aprender-contracts-cli, and -- until paiml#2553 -- the facade. The operator settled that this binary KEEPS the name (2026-08-21), which makes the version line the mitigation the project relies on. It carried no identity at all. The install-time half of the collision was already handled (check_duplicate_bin_names.sh, the facade going lib-only); nothing answered the RUNTIME question "which one am I talking to". pv 0.63.0 (aprender provable-contracts verifier) crate aprender-contracts-cli - https://github.com/paiml/aprender Verifies YAML contracts under contracts/; run `pv --help` for the command surface. This is NOT pv(1), the pipe viewer (distro package `pv`, or the `pv` crate on crates.io). `-V` stays one line and is itself unambiguous: a version line nobody reads is no better than an ambiguous one. ADDING IDENTITY TEXT BROKE THE RELEASE GATE, and the fix ships in the same commit. scripts/pv_bin.sh proves a resolved pv is the HEAD build by extracting the semver with `awk '{print $NF}'` -- the last field of EVERY line. Against a multi-line version it returned four lines of prose and pv_bin.sh exited 1, blocking the Makefile contracts gate and dogfood_surfaces.sh. The extractor now reads position 2 of line 1, and that position is pinned from BOTH sides so neither can drift alone. FALSIFIER, RED BEFORE THE FIX (rc=101): 5 of 8 integration assertions and 3 of 5 unit assertions fail against `pv 0.63.0`. Mutation-verified in both directions, six mutations, each with its engagement proved by reading back the mutated line: M1 bare `version` -> 5/8 integration RED M2 drop only the disclaimer -> exactly 1 RED (the clause is load-bearing) M3 semver off field 2 in -V -> the agreement assertion RED M4 semver off field 2 in --version -> the pv_bin.sh contract RED, and pv_bin.sh itself exits 1 -- guard and gate fail as one M5 M1 re-run in LIBRARY scope -> 3/5 RED (the old proof does not transfer) M6 drift pv_bin.sh's extractor -> EXTRACTOR_MISMATCH fires The assertions live in the LIBRARY as well as in tests/, because a new tests/*.rs file is dark until named on ci.yml's single explicit --test line and only one PR at a time can edit it. `--lib` reaches the unit half unconditionally. The new case table caught its own comment-vs-code trap on first run: it grepped all of pv_bin.sh and fired on pv_bin.sh's COMMENT quoting the old extractor. Fixed to strip full-line comments. Verified: pv lint contracts/ 0 errors, 947 warnings -- identical with and without the new contract, so it adds none (the 942 figure in circulation is stale; re-measured here). fmt clean, clippy -p aprender-contracts-cli --lib -D warnings clean, bashrs 0 errors/0 warnings on the new script, shell-lint ratchet PASS at 97 error lines. Contract: contracts/pv-version-identity-v1.yaml, FALSIFY-PVVER-001..007, all seven executed and passing. Refs paiml#2559 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR * fix(pv): close the eight defects an adversarial pass measured on paiml#2601/paiml#2602/paiml#2603 Three PRs were opened and auto-merge armed; both verifiers returned sound=false. This folds all three onto one branch and closes every finding. Every number below was RE-MEASURED on this branch -- none was copied from the review brief, and two of the brief's numbers were wrong. F1 (FATAL) required check guard-runner-labels check_readme_claims.sh was rc=1: the corpus goes 1778 -> 1781 (THREE new contracts land, one per folded PR -- the brief said 1779, assuming one). README:44/:225/:256 bumped; script now rc=0. M1 both whole-surface tests were NEAR-VACUOUS MEASURED: replacing Commands::Status's body with panic!() -- so `pv status` dies on every invocation -- left ALL 8 tests GREEN. clap answers `--help` without entering the command, so the reachability test proved only that the command was DECLARED; and a panic exits 101, which the rejection test read as a correct refusal. The test could not detect the class (paiml#2527) its own doc comment claimed. FIX: every contract-taking subcommand is now INVOKED FOR REAL against a valid contract (17 of them, run in a scratch cwd because the generators write into ./generated/), and a rejection must be a clean 1 or 2, never a panic. Three mutations, each proved engaged by grepping its marker in the built binary before any verdict was read: panic!() 8 pass/0 fail -> 6 pass/2 fail exit(1) on valid 8 pass/0 fail -> 7 pass/1 fail exit(0) on all 8 pass/0 fail -> 7 pass/1 fail All restored; `grep -c MUTANT` is 0. M2 the two collisions between the folded PRs (a) TEXTUAL: paiml#2559 and paiml#2589 both append to the single integration-test line in ci.yml. Resolved in-tree keeping BOTH -- pv_surface_gate, cli_integration, ground_truth, version_identity. (b) EXHAUSTIVENESS: paiml#2589's guard reds the moment paiml#2555 lands. Reproduced verbatim: ["CRUX-001", "CRUX-002"] have no row. Closed with REAL CLI trip cases, not bookkeeping entries -- each asserts `[ERROR] CRUX-00N:` and rc=1 through the binary. M3 4 of 7 falsification commands were NOT EXECUTABLE as written -003/-004/-006 passed 2-3 test filters as bare positionals (cargo exits 1, "unexpected argument"); -007 embedded prose in the command (bash exits 2, syntax error). Multi-filter args moved after `--`; -007's protocol prose moved into `prediction` where it belongs. All 9 now run verbatim at rc=0, read OUT OF THE YAML rather than retyped, with filter counts proving each ran the tests it names. m1 the headline rationale was unsupported by the code paiml#2555 justified CRUX-001 as guarding "the ranking signal the whole competitive-research programme sorts by". MEASURED: nothing in the repo reads metadata.demand_score. The signal spec section 12.1 maps to `pmat work` priority is stories[].demand_score in the MASTER REGISTRY -- 250 rows, entirely ungated. Rather than retreat to a weaker claim, the gate now covers that surface too, reusing DEMAND_SCORE_RANGE / CRUX_COMPETITORS / IntakeStatus, so the claim is true of the union. On a registry row the fields are REQUIRED as well as bounded. Lands GREEN (250 rows all in-domain) and functions as a ratchet. m2 trim laundering, and the presence gap CRUX-002 trimmed INSIDE the comparison, so `competitor: ' ecosystem '` validated clean while the stored value kept its padding -- the gate approved a string no consumer would see. Normalisation moved to PARSE. Present-but- empty deliberately does NOT collapse to None: that would have widened the presence gap rather than narrowed it. The gap that remains (omission and explicit null dodge the metadata surface entirely, and cannot be closed there because ~1450 non-crux contracts carry no such fields) is now a stated proof obligation, not an unnoticed hole. m3 stale counts in files whose thesis is that unmeasured claims rot "all 18 diagnostic rules" -> 27 (measured; was 25 pre-paiml#2555, never 18). "276 contracts carry competitor" -> 275 FILES / 292 DECLARATIONS (17 crux contracts carry a second competitor inside an equivalence obligation), so the BEAT matcher accepts 58 of 292 and rejects 80%, not 79%. m4 the identity was not load-bearing where the decision is made paiml#2559 added an identity string to `pv --version`, but pv_bin.sh -- the place the release actually decides it has the right binary -- still proved freshness from the SEMVER alone. pv_bin_assert_fresh now asserts identity on the same line it already parses. Mutation-verified: a fake pv printing the exactly-correct declared version `pv 0.63.0` and nothing else is now REJECTED (rc=1); the real binary passes. The restructure kept the extractor literal that check_pv_version_parse.sh pins, and that guard plus its --self-test and check_sourced_libs_option_neutral.sh are all rc=0. m5 residual stated as a FRACTION "tranche" is honest about being partial and silent about how partial. The scope block now gives three concentric rings: DEEP behavioural decision table 1/38 subcommands (`pv validate` alone), REAL INVOCATION 17/38, SURFACE 38/38. The behavioural depth of this tranche is 1 of 38. VERIFICATION check_readme_claims.sh rc=0 check_facade_compat.sh rc=0 (see PR body: rc=1 inside this worktree is dev-config inheritance, proved by running the same tree outside /home/noah/src/aprender) check_pv_version_parse.sh / --self-test rc=0 / rc=0 check_sourced_libs_option_neutral.sh rc=0 bashrs lint scripts/pv_bin.sh 0 errors cargo fmt --all -- --check rc=0 aprender-contracts --lib 1466 passed, 0 failed aprender-contracts-cli (all targets) 182 passed, 0 failed (book_coverage excepted -- byte-identical pre-existing failure on origin/main: contracts/binding.yaml, "missing field metadata") pv lint contracts/ 0 errors, 947 warnings, PASS -- IDENTICAL to the origin/main baseline I measured myself (947, not the 937 or 942 two earlier briefs asserted); 1727 -> 1730 contracts, 0 new warnings 9/9 FALSIFY-CRUX-INTAKE-* verbatim rc=0 Closes paiml#2555 Closes paiml#2559 Closes paiml#2589 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR * fix(pv): close the three minor defects the re-verification pass measured The consolidated branch was sound on every FATAL and MAJOR item. Three minor findings survived, and all three are the same shape: a claim that no longer matches what was measured. 1. A stale percentage in the file whose thesis is that unmeasured claims rot. contracts/crux-intake-metadata-domains-v1.yaml:78 was correctly re-measured to "58 of 292 ... reject 80%", but :232 still said 79%. Swept the whole folded diff for the superseded numbers (276 contracts, 79%, "all 18 diagnostic rules", 16 rules) — this was the only survivor. 2. The residual disclosure overstated its own coverage, which is precisely the defect class m1 was raised to close, recurring one level up. The NOT COVERED paragraph said the other 37 subcommands are asserted "to run and decide", but the ring table above it says only 17/38 are invoked for real. The other 21 are asserted only to be DECLARED. Rewritten to state both tiers separately and to say plainly that a `pv lint`, `diff`, `query`, `graph`, `kaizen` or `migrate` whose body were panic!() would leave the file GREEN. 3. cli_integration dirties TRACKED generated caches (.pv/contracts.idx, .pv/contracts.idx.mtime, .pv/lint-previous.json), and this PR wires it into ci.yml's integration step — so CI runs it for the first time and its working tree goes dirty. Disclosed at the top of the file rather than contained, with the reason: pv_surface_gate solves this with pv_in(dir, ..) + .current_dir(), which does not transfer, because cli_integration makes IN-PROCESS library calls and the cache path follows the test process's own cwd. Changing that cwd is process-wide and would race the other tests in the binary. A proper fix means an explicit cache-root parameter on the library, which is a real change and not this PR's. Verified: pv_surface_gate 8/8 pass, check_readme_claims.sh rc=0 (contract_count 1781), pv validate on the edited contract rc=0 / 0 errors 0 warnings. Built with an explicit --target-dir, not the env var, per the shared-target-dir trap. Refs paiml#2555, paiml#2559, paiml#2589, paiml#2556 * chore(pv): re-measure the contract count after batch-b landed paiml#2631 folded ten PRs, six of which each add one contract, taking main 1781 -> 1787. This branch adds three of its own, so the true corpus is 1790. Both branches were correct in isolation and neither could see the other — the paiml#2630 collision, now hit for the sixth time in two days. The figure is MEASURED here (find contracts -name '*.yaml'), never incremented by guesswork, because the right value depends on what else landed and that is unknowable from inside a branch. check_readme_claims.sh rc=0: crate_count 78, contract_count 1790, cli_command_count 110, cookbook_link present. * fix(guard): check_pv_bin_resolution.sh read the identity suffix as part of the semver MERGE SEAM, not a defect in either side. paiml#2559 (this branch) appended an identity suffix to `pv --version` because four things claim the name `pv` and the operator settled 2026-08-21 that this binary keeps it, so the version line IS the disambiguation. paiml#2580 (main, batch-a) added this guard, whose own extraction compared `--version | head -1` against the literal "pv $CRATE_VERSION". The suffix rode along into the compared value and the guard went RED against a perfectly fresh HEAD build: FAIL: $PV reports 'pv 0.63.0 (aprender provable-contracts verifier)' but the crate is 0.63.0 pv_bin.sh's own pv_bin_assert_fresh() was updated by paiml#2559; this script carries a SEPARATE extraction that was not. Two changes, not one: 1. Semver read POSITIONALLY -- position 2 of line 1, the shape pinned from the other side by tests/version_identity.rs and by the case table in check_pv_version_parse.sh -- instead of whole-line equality. 2. The guard now asserts the IDENTITY too. Fixing only the extraction would leave a guard that passes for pv(1) the pipe viewer, since a semver is not an identity; the identity is the property paiml#2559 exists to provide, so the guard that runs the resolver has to assert it. The literal is checked against both aprender-contracts-cli/src/lib.rs and pv_bin.sh (IDENTITY_MISMATCH), so the guard cannot end up proving a string nobody emits. Case table extended in both directions, per the file's existing REFUSE/ACCEPT pairing: 2c REFUSE: a pv at the CORRECT crate version printing the bare `pv <semver>` -- byte-for-byte what pv(1) prints -- and the refusal must be on identity grounds, not incidental. 2d ACCEPT: the same synthesized script, same version, plus the suffix. The only difference between the pair is the property under test. MUTATION EVIDENCE (both directions, engagement proven): * pre-fix file, unmodified: rc=1, the exact CI message above. * M1 -- delete the identity `case` block from pv_bin.sh (engagement: `grep -c 'aprender provable-contracts verifier' scripts/pv_bin.sh` 1 -> 0): rc=1, 2c RED ("ACCEPTED a bare 'pv 0.63.0' with no identity") plus IDENTITY_MISMATCH, while controls 2b and 2d stayed green -- so the pair discriminates rather than refusing everything. * M2 -- regress the new extractor to the pre-paiml#2559 `{print $NF}` (engagement: the line read `got_semver=$(awk '{print $NF}' ...)`): rc=1, semver read as 'verifier)'. * fixed file: rc=0, 10/10 assertions. OTHER CONSUMERS of `pv --version`, checked: scripts/dogfood.sh:334 reads `head -1 | awk '{print $2}'` (positional, safe); scripts/dogfood_surfaces.sh:258 uses `-V`, the deliberately one-line glance form (verified single-line against the HEAD build); pv_surface_gate.rs asserts a substring. None break. bashrs: 0 errors on this file alone; check_shell_lint_ratchet.sh PASS (145 error lines vs baseline 876). Refs paiml#2617, paiml#2559, paiml#2580 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR * fix(pv): gate SCHEMA-018/019/020 through the CLI, not with a bookkeeping entry paiml#2589 added an exhaustiveness guard requiring every rule declared in validator.rs to have a row in the pv-surface decision table. paiml#2554 landed SCHEMA-018/019/020 on main while this branch was open. The merge brought main's rules into a tree whose guard demands rows for them, so workspace-test went RED with: validator.rs declares rules with no row in the pv-surface decision table: ["SCHEMA-018", "SCHEMA-019", "SCHEMA-020"] That is a MERGE SEAM, not a defect in either side, and the guard is right. ALSO_ABSENT_FROM_BASELINE was deliberately not used. paiml#2589 exists because contracts-pv had 76 features at 0% coverage; satisfying its own guard with a bookkeeping entry would make the ticket self-defeating. This is the same resolution CRUX-001/002 got when they collided in the other direction: three REAL trip cases driving the pv BINARY against contracts that actually carry the offending shape, asserting the rule id and a non-zero exit. Each rule is tripped on its OWN terms, read out of `validate_top_level_keys`. They share a function but not a mechanism, so a single fixture would have proved one rule three times: SCHEMA-018 a literal top-level `kind:`. Value is `KernelContract`, the exact string the 72 mislabelled registry contracts carried, so a pv that READ the key instead of reporting it would also accept it. SCHEMA-019 a NEAR-MISS of a real block name — `falsification_test:`, the singular. Different arm: unknown AND `near_miss_of` resolves it. This row is what makes 018 and 019 discriminable through the CLI; a pv reporting every unknown key under one id fails whichever it did not print. SCHEMA-020 a DUPLICATE MAPPING KEY, which fires off `strict_yaml_error`, not the unknown-key loop at all. The duplicate is nested inside an UNKNOWN top-level block on purpose: a duplicated *known* key is a hard serde "duplicate field" parse error rather than SCHEMA-020, and `commands:` is not a near-miss of any contract field, so the row cannot be satisfied by a pv emitting SCHEMA-019 instead. It reproduces contracts/apr-cli-commands-v1.yaml, the file the rule was born from. Tripping these needs a top-level SIBLING of `metadata:`, not an edit to a field the schema knows, so `Fixture` gains an `extra` slot rendered after `qa_gate:`. Empty by default, which preserves the one-swap-per-case rule. MEASURED, not assumed. Each fixture run directly against the binary reports `1 error(s), 0 warning(s)` and rc=1 — exactly one rule each, so attribution is clean and the three are mutually exclusive. MUTATION-VERIFIED, both directions, mutation proven to have engaged each time: drop the three rows -> every_rule_in_the_validator_source_appears_in_the_table RED with the CI message byte-for-byte `kind:` -> `family:` -> SCHEMA-018 RED ("expected a `[ERROR] SCHEMA-018` line", pv printed "Contract is valid") near-miss -> plain key -> SCHEMA-019 RED drop the duplicate key -> SCHEMA-020 RED put the block in Default -> validate_baseline_is_silent_and_exits_zero RED, i.e. the negative direction still discriminates Restored file md5 identical to the pre-mutation copy; all four CI-wired pv integration targets green (114 tests). Header counts re-measured: 27 -> 30 diagnostic rules. Refs paiml#2589, paiml#2554 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbMTnT8Upx6Ym18i5H11iR --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

noahgift and others added 2 commits August 21, 2026 09:48

noahgift force-pushed the fix/pv-schema branch from 08bc73b to 393037f Compare August 21, 2026 07:48

noahgift mentioned this pull request Aug 21, 2026

Epic: pv / provable-contracts hardening — the tool that decides release soundness #2556

Open

8 tasks