Repository navigation
Harden compiler bootstrap against compromised certificate authorities - #70549
Conversation
Authenticate release checksum manifests with trusted action pins, enforce HTTPS-only bootstrap redirects, require patched Go release toolchains, and document runtime trust ownership and emergency distrust procedures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Commit pushed:
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Request changes
The Docker Compose bootstrap pins the binary bytes, but then loads the verifier implementation from the job-writable RUNNER_TEMP tree. That allows earlier workflow code to replace the verifier and bypass the pin entirely. The inline comment describes the required trust-boundary fix.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 3.46 AIC · ⌖ 0.608 AIC · ⊞ 1.6K
Comment /review to run again
🏗️ Design Decision Gate — ADR RequiredThis PR makes significant changes to core business logic (272 new lines in 📄 Draft ADR committed: The draft is based on the diff. It records the decision to authenticate AWF/Copilot checksum manifests against release-bound SHA256 pins shipped in the trusted setup action (
📋 What to do next
Once an ADR is linked in the PR body, this gate will re-run and check that the implementation matches the decision. Reviewer note (unverified): the "incident report" link in the PR body and in ❓ Why ADRs MatterADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you. 📋 Michael Nygard ADR Format ReferenceAn ADR needs these four sections to be complete:
All ADRs are stored in
|
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /improve-codebase-architecture and /codebase-design (per pr-triage: architecture_change). This is a well-scoped, well-tested security hardening PR — I verified the digest-pin logic, HTTPS-enforcement, and the CA-compromise regression tests (tls_trust_conformance_test.go) by reading the code and running release_integrity_test.sh locally (10/10 assertions pass). No correctness defects found; feedback below is about test-coverage consistency and pin-maintenance tooling, not blockers.
📋 Key Themes & Highlights
Key Themes
- Test coverage asymmetry: AWF, Copilot, and threat-detect installers all gained dedicated test scripts for their new digest-verification paths in this PR;
install_gh_cli.sh's new pinned apt-key verification did not. - Pin-maintenance tooling gap:
update-threat-detect-pins.shandupdate_container_pins.goexist to generate/validate their pin lists;release_integrity.sh's AWF/Copilot heredoc has no equivalent, so it's the one pin table still maintained by hand-editing. - One inline comment (on
security-architecture-spec.md) was based on a misreading on my part — TT-01 already names the GitHub CLI key installer and generated Compose installer explicitly. Please disregard that specific comment; I'm leaving it visible for transparency rather than silently dropping it.
Positive Highlights
- ✅
release_artifact_sha256()'s fail-closed design (missing/duplicate/malformed pin allexit 1, no fallback) is exactly right for this threat model. - ✅
tls_trust_conformance_test.gouses real, legitimately-public certificates to reproduce the cross-signed compromised-CA scenario — a strong, concrete regression test rather than a mocked abstraction. - ✅
scripts/check-go-toolchain.sh+build-release.sh's embedded-toolchain verification closes the loop between the stated floor and what's actually shipped in release binaries.
@copilot please address the review comments above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 169.1 AIC · ⌖ 15.1 AIC · ⊞ 10.2K
Comment /matt to run again
Add engine.checksums-sha256 and sandbox.agent.checksums-sha256 with literal version binding, propagate pins through detection and eval installers, and verify explicitly pinned Copilot host and musl downloads without toolcache bypasses. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve the remote architecture decision record and align it with explicit workflow checksum-manifest pins and cache verification semantics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Security-critical release pins require independent maintainer verification, and bootstrap compatibility issues remain unresolved.
5 open issues
What changed in this PR
Hardens gh-aw’s compiler and setup bootstrap by separating HTTPS transport checks from independent artifact authentication.
Changes:
- Adds trusted digest verification and HTTPS-only bootstrap downloads.
- Raises Go toolchain floors and adds runtime diagnostics.
- Expands regression coverage, security specifications, and migration guidance.
| File | Description |
|---|---|
specs/security-architecture-spec.md |
Defines transport-trust controls and ownership. |
specs/compiler-threat-detection-spec.md |
Maps bootstrap controls to compiler boundaries. |
specs/compiler-threat-detection-changelog.md |
Records specification changes. |
scripts/update-threat-detect-pins.sh |
Restricts maintenance downloads to HTTPS. |
scripts/check-go-toolchain.sh |
Enforces patched stable release toolchains. |
scripts/check-go-toolchain_test.sh |
Covers toolchain acceptance and rejection. |
scripts/build-release.sh |
Verifies and records build toolchains. |
pkg/workflow/tls_trust_conformance_test.go |
Tests CA distrust and redirect rejection. |
pkg/workflow/runtime_setup_test.go |
Updates expected Go default. |
pkg/workflow/runtime_integration_test.go |
Uses the patched Go default constant. |
pkg/workflow/firewall_version_pinning_test.go |
Checks default pins and Compose integrity. |
pkg/workflow/copilot_engine_installation.go |
Verifies Compose bytes before installation. |
pkg/constants/version_constants.go |
Raises default Go to 1.26.10. |
pkg/cli/install_copilot_cli_test.go |
Updates installer fixtures and rejection checks. |
Makefile |
Includes bootstrap hardening shell tests. |
go.mod |
Raises minimum and preferred Go toolchains. |
CHANGELOG.md |
Documents breaking changes and migration. |
actions/setup/sh/runtime_trust_inventory.sh |
Adds diagnostic host TLS inventory. |
actions/setup/sh/release_integrity.sh |
Provides trusted pins and digest verification. |
actions/setup/sh/release_integrity_test.sh |
Covers tampering and unreviewed releases. |
actions/setup/sh/install_threat_detect_binary.sh |
Restricts binary downloads to HTTPS. |
actions/setup/sh/install_threat_detect_binary_test.sh |
Checks HTTPS-only download flags. |
actions/setup/sh/install_gh_cli.sh |
Authenticates apt signing-key bytes. |
actions/setup/sh/install_copilot_cli.sh |
Authenticates manifests before archive downloads. |
actions/setup/sh/install_awf_binary.sh |
Authenticates manifests and rejects integrity fallback. |
actions/setup/sh/install_awf_binary_test.sh |
Updates checksum and download-policy fixtures. |
actions/setup/sh/cloud_hypervisor_setup_enclave_artifacts.sh |
Restricts installer downloads to HTTPS. |
actions/setup/sh/cloud_hypervisor_setup_bundle.sh |
Restricts bundle downloads to HTTPS. |
actions/setup/setup.sh |
Invokes runtime TLS inventory during setup. |
.changeset/compiler-transport-trust.md |
Declares the major compatibility change. |
Review effort: Max
…ggles Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inline Compose verification under Bash, add isolated signing-key and diagnostic regressions, automate independently authenticated release-pin authoring, preserve reviewed musl guest cache compatibility, migrate the smoke workflow, and clarify the ADR and security conformance scope. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
In reply to Test Quality Sentinel: no code change is needed because this completion notice contains no concrete finding. Its reported_no_action outcome is not being treated as evidence that validation or publication is complete. |
|
In reply to Ponytail Reviewer: this completion notice includes no actionable finding requiring another change. Reviewer completion alone does not establish that all required work is complete. |
|
In reply to Design Decision Gate: reviewed the generated ADR, replaced verification placeholders with bounded evidence, and linked it explicitly in the PR body. It remains Draft pending maintainer acceptance. |
|
In reply to PR Code Quality Reviewer: the associated Compose trust-boundary finding is fixed with inline digest verification and regression coverage. This completion notice requires no additional code change. |
|
In reply to Matt Pocock Skills Reviewer: addressed the actionable inline feedback on signing-key tests and pin maintenance, and clarified specification wording without treating the withdrawn complaint as a missing control. No separate change is needed for this completion notice. |
|
In reply to the ADR requirement: completed the verification placeholders and added an explicit ADR link to the PR body, while retaining Draft status rather than inventing maintainer approval. The ADR cites primary Mozilla and Go announcements, labels the incident website as secondary, and records the bounded attestation evaluation and remaining Maven limitation. |
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: a9dd386
|
…ggles Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Merged latest |
Resolve the overlapping portfolio test updates using parsed activation conditions, retaining rerun and admission checks independently of YAML formatting. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: 07834bc
|
|
/smoke-copilot |
|
📰 BREAKING: Smoke Copilot is now investigating this issue comment. Sources say the story is developing... |
The Go 1.27 toolchain moves HTTP/2 read loops into net/http/internal/http2. Close the default client's idle connections after tests finish instead of adding leak-detector exclusions for the new implementation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merged latest |
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: d4db889
|
Upgrade module, runtime, smoke, lint and devcontainer toolchain pins; refresh Go roots without inherited GOROOT for host builds and Serena; add baseline and stale-root regressions and retain only required Go 1.27 diagnostic/initializer compatibility fixes. Validation: targeted Go tests, build, formatting, WASM build and all 345 workflow recompilations passed. Full lint checkpoint remains blocked by existing custom-analyzer diagnostics and unknown-linter warnings; no checks were disabled. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Update the smoke-copilot fixture to include strict, stale-root-safe GOROOT discovery. Golden regeneration and verification passed; the repository-wide checkpoint retains its previously reported lint blockers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merged latest |
Why
The October 2026 eMudhra intermediate CA compromise exposes a gap between destination allowlisting and endpoint authentication: an intercepting attacker can replace both an executable and its network-served checksum. Primary response sources are the Mozilla root-program announcement and Go security announcement; the incident website is a secondary report.
ADR: ADR-70549: Authenticate Bootstrap Downloads with Release-Bound Digests in the Setup Action
The ADR's verification placeholders are complete; it remains Draft pending maintainer acceptance.
Approach
sandbox.agent.checksums-sha256/engine.checksums-sha256frontmatter. Supplied pins require explicit versions, propagate through matching detection/eval installations, and fail closed on malformed or mismatched input. Copilot overrides bypass toolcache and authenticate both host and requested musl archives.Compatibility and limitations
Breaking change: Downloads without a built-in or explicitly supplied manifest pin and uncached
latestdownloads fail closed. Supplied pins authenticatechecksums.txt/SHA256SUMS.txt, not individual binaries, and must come from an independently trusted channel. Update/recompile workflows to use the new setup action. Repository builds and tooling require Go 1.27.3; rebuild existing devcontainers to install the pinned feature.This PR does not deploy releases or rebuild upstream proxy/MCP/container/guest trust stores. Inline Maven bootstrap remains outside the hardened HTTPS-only and independent archive-integrity guarantees. Inventory versions are not trust-store attestations; broader package-manager enforcement remains a candidate, not an active compiler threat rule.
Validation