Skip to content

feat(release): 2 of 10 bundled artifacts were signed but undescribed for 58 releases - #375

Merged
avrabe merged 3 commits into
mainfrom
feat/bundle-manifest-describes-attested
Sep 2, 2026
Merged

feat(release): 2 of 10 bundled artifacts were signed but undescribed for 58 releases#375
avrabe merged 3 commits into
mainfrom
feat/bundle-manifest-describes-attested

Conversation

@avrabe

@avrabe avrabe commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Verify-Filter: (has-tag "manifest")

Filter narrowed from (or (has-tag "manifest") (has-tag "release")). The
broader form selected FV-FALCON-RELEASE-001, whose three gh / cosign
steps exit rc=127 on the gate runner and had never been executed — that
artifact carries no falcon tag, so the default sweep never matched it. That
is a real latent defect, not something to filter around, and it is fixed in
#374 by classifying release-verification commands bench-only by shape
(4 steps across 2 artifacts). This PR narrows to the artifact it actually
introduces; once #374 lands the broader filter would pass too.

jess asked on #202 which cascade artifact to fuse. The bundle could not answer.

Measured on the published artifact

falcon-components-v1.135.tar.gz, pulled from the falcon-v1.135.0 release:

count
entries in the cosign-signed SHA256SUMS 10
components declared in manifest.json 8
signed but undescribed 2
falcon-cascade-stream-composed-v1.135.wasm
falcon-cascade-stream-fused-v1.135.wasm

Why that's an ambiguity and not untidiness: one directory holds
falcon-cascade-v1.135.wasm — the cargo-component cascade that gets fused and
lowered — beside falcon-cascade-stream-{composed,fused}-v1.135.wasm, which are
P3 async stream variants: a different artefact class with a near-identical
name
. Asked "which of these did you verify, and which do I fuse?", a consumer
had only the filename, because the machine-readable manifest listed neither.

The attestation was never broken

Stated explicitly because which failure this is matters. Inside the unpacked bundle:

$ shasum -a 256 -c SHA256SUMS
./falcon-attitude-v1.135.wasm: OK
... all 10 ...
exit 0

So SWREQ-RELAY-STREAM-P15 is genuinely satisfied — the stream artifacts are
bundled and are covered by the signature. What drifted, silently, across 58
releases is the bundle's self-description, because nothing compared the two files.

Fix: describe, don't remove

manifest.json now carries kind:

{ "name": "falcon-cascade", "file": "falcon-cascade-v1.136.wasm",
  "kind": "flight-component",   "sha256": "...", "bytes": ... }
{ "name": "falcon-cascade-stream-fused", "file": "falcon-cascade-stream-fused-v1.136.wasm",
  "kind": "p3-stream-artifact", "sha256": "...", "bytes": ... }

Telling them apart becomes a field lookup instead of a guess from a filename.

Removal was the originally agreed fix and is rejected on evidence: it would
contradict SWREQ-RELAY-STREAM-P15 (which requires precisely that they be
bundled and attested), falsify FV-RELAY-STREAM-015, and would not answer the
question actually asked — which is "what IS this file", not "why is it here".

Drift guard, tested both ways

negative (the exact v1.135 shape, reproduced deliberately):
  ERROR: signed by SHA256SUMS but undescribed in manifest.json:
         falcon-cascade-stream-fused-v1.136.wasm            exit 1
positive:
  manifest/SHA256SUMS agree on every bundled artifact       exit 0

A guard only ever seen to pass is indistinguishable from one that cannot fail, so
the negative case is the bug that motivated it, not a synthetic stand-in.

A correction worth recording

This was first mis-diagnosed as something far worse — "9 of 10 signed files are
missing from the release"
. Listing release assets through a name filter of
wasm|SHA256|manifest excludes falcon-components-v*.tar.gz, whose name contains
none of those; checking each SHA256SUMS entry against that already-filtered list
then reported 9 MISSING. The tarball was there the whole time — the filter was
applied twice, and the second use inherited the first one's blind spot.

The real defect is smaller, different in kind, and was only visible after
unpacking the actual published artifact rather than reading the script that
produces it.

Evidence

rivet validate                                       exit 0
FV-FALCON-REL-002: 4/4 steps PASS
shasum -c SHA256SUMS (published v1.135 bundle)       10/10 OK, exit 0
guard negative / positive                            exit 1 / exit 0

Two-commit rule: this PR carries code, so nothing goes past implemented.

Refs #202

🤖 Generated with Claude Code

https://claude.ai/code/session_01HvusAXYbHLyv3uTzfBcMbG

… 58 releases

jess asked on #202 which cascade artifact to fuse. The bundle could not answer:
manifest.json declared EIGHT components while the cosign-signed SHA256SUMS
covered TEN. The two undescribed ones were the P3 stream artifacts.

MEASURED ON THE PUBLISHED ARTIFACT, not on the script that makes it —
falcon-components-v1.135.tar.gz pulled from the falcon-v1.135.0 release:

  entries in the signed SHA256SUMS       10
  components declared in manifest.json    8
  signed but UNDESCRIBED                  2
      falcon-cascade-stream-composed-v1.135.wasm
      falcon-cascade-stream-fused-v1.135.wasm

Why that is an ambiguity and not untidiness: in one directory sit
`falcon-cascade-v1.135.wasm` — the cargo-component cascade that gets fused and
lowered — and `falcon-cascade-stream-{composed,fused}-v1.135.wasm`, P3 async
stream variants, a different artefact class with a near-identical name. Asked
"which of these did you verify, and which do I fuse?", a consumer had only the
filename, because the machine-readable manifest listed neither.

THE ATTESTATION WAS NEVER BROKEN, and which failure this is matters. Inside the
unpacked bundle `shasum -a 256 -c SHA256SUMS` passes 10/10, exit 0. So
SWREQ-RELAY-STREAM-P15 is genuinely satisfied — the stream artifacts are
bundled and covered by the signature. What drifted, silently, for 58 releases
is the bundle's SELF-DESCRIPTION, because nothing compared the two files.

FIX: describe, do not remove. manifest.json now carries `kind` —
`flight-component` for the eight cargo-component stages, `p3-stream-artifact`
for the two bazel-built variants — so telling them apart is a field lookup, not
a guess from a filename.

Removal was the originally agreed fix and is rejected on evidence: it would
contradict SWREQ-RELAY-STREAM-P15 (which requires precisely that they be
bundled and attested), falsify FV-RELAY-STREAM-015, and would not answer the
question actually asked — which is "what IS this file", not "why is it here".

DRIFT GUARD, tested BOTH ways. build-components.sh now fails the build when a
SHA256SUMS entry is undeclared:

  negative (the exact v1.135 shape, reproduced deliberately):
    ERROR: signed by SHA256SUMS but undescribed in manifest.json:
           falcon-cascade-stream-fused-v1.136.wasm            exit 1
  positive:
    manifest/SHA256SUMS agree on every bundled artifact       exit 0

A guard only ever seen to pass is indistinguishable from one that cannot fail,
so the negative case is the bug that motivated it, not a synthetic stand-in.

A CORRECTION WORTH RECORDING. This was first mis-diagnosed as something far
worse — "9 of 10 signed files are missing from the release". Listing release
assets through a name filter of "wasm|SHA256|manifest" excludes
falcon-components-v*.tar.gz, whose name contains none of those; checking each
SHA256SUMS entry against that already-filtered list then reported 9 MISSING.
The tarball was there the whole time. The filter was applied twice and the
second use inherited the first one's blind spot. The real defect is smaller,
different in kind, and only visible after unpacking the published artifact.

Two-commit rule: this PR carries code, so nothing goes past `implemented`.
rivet validate exit 0; FV-FALCON-REL-002's 4 steps PASS.

Refs #202

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvusAXYbHLyv3uTzfBcMbG
@avrabe
avrabe enabled auto-merge (squash) September 2, 2026 11:33
avrabe added a commit that referenced this pull request Sep 2, 2026
…sible

Found by the gate failing on PR #375, which is how it should have been found
three years of releases earlier.

FV-FALCON-RELEASE-001 has carried three steps since v0.17.0:

  gh release view falcon-v0.17.0 --json assets
  cosign verify-blob --bundle SHA256SUMS.txt.cosign.bundle SHA256SUMS.txt
  gh attestation verify falcon-v0.17.0-...tar.gz --repo pulseengine/relay

All three exit 127 on the gate runner — `gh` and `cosign` are not installed.
None had ever been executed, because the artifact is tagged
[release, supply-chain, cosign, sbom, slsa, v0.17.0] with NO `falcon` tag, so
the default `(has-tag "falcon")` sweep never selected it. It took a broader
Verify-Filter on an unrelated PR to surface all three at once.

FV-RELAY-STREAM-014 carries a fourth, same shape, same reason.

This is the SAME TRAP FV-FALCON-REL-001 documented and deliberately designed
around — quoting its own steps block: a `gh release view <tag>` step "would
really execute and fail with rc=1 on every PR until the release exists, and
then pass for the rest of time. That is a step whose result depends on WHEN it
runs, which is not evidence." REL-001 avoided it by hand. Nothing stopped the
next author walking into it, twice.

FIX: classify by COMMAND SHAPE, which is how every other bench-only decision
here is made — `is_bench_only` consults BENCH_PATTERNS only; the trailing
`# bench-only` comment is decorative and the module docstring saying otherwise
is stale. Two patterns added:

  ^\s*gh\s+(?:release|attestation)\s      needs gh + a PUBLISHED release
  ^\s*cosign\s+(?:verify|verify-blob)\b   needs cosign + published signatures

Anchored at line start, so a step that merely NAMES `gh release` in a longer
pipeline is not silently skipped — the same anchoring bug that made
FV-FALCON-OCI-004 ship verifying nothing in v1.134.

OVER-MATCH CHECKED, not assumed: swept every step in every artifact, exactly 4
newly classify as bench-only, and all 4 are the release-verification steps
above. Nothing else in the tree matches.

  before: [ FAIL (rc=127)] x3   exit 1
  after:  [skip-bench-only] x3  exit 0

Two-commit rule: code change, nothing promoted past `implemented`.

Refs #375

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvusAXYbHLyv3uTzfBcMbG
@avrabe
avrabe merged commit 34cc0fd into main Sep 2, 2026
11 checks passed
@avrabe
avrabe deleted the feat/bundle-manifest-describes-attested branch September 2, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant