Skip to content

fix(cache): lane compile-cache writes per pull request and bound them at admission - #239

Merged
debuggingfuture merged 1 commit into
mainfrom
gate/compile-cache-hardening
Oct 23, 2026
Merged

debuggingfuture merged 1 commit into
mainfrom
gate/compile-cache-hardening

Conversation

@debuggingfuture

Copy link
Copy Markdown
Member

Follow-up to #238. Pull-request code holds its compile-cache token, so the cache must not trust what a pull request writes. This PR confines a pull request's writes to that pull request and enforces the byte bounds when a write is admitted.

Problem & Insight

  • Cross-PR poisoning. In feat(cache): shared sccache compile cache for contextful gate parts #238 every token could read and write one shared prefix per repository. A same-repository PR could PUT a forged entry under the deterministic sccache hash of a stable input, such as a crates.io rlib. Later gates on other PRs and on the default branch would then link and run it for up to 14 days.
  • Unbounded writes. PUT admission read a size the hourly sweep measured. A token copied out of the container stayed valid after its execution ended, so a loop of unique keys could write without limit until the next sweep.
  • Sweep blind spot. The sweep stopped at 500 pages and restarted from an empty cursor every hour. Keys past the cap were never expired, evicted or measured, and a repository first seen there was admitted against a size of 0.
  • Eviction by write time. Eviction removed the oldest-written entries first, which are the dependency rlibs every gate reads.

Take

  • Lanes, bound into the token's MAC (cc2). The lane is base (default-branch executions), pr-<n> or read-only. pr-<n> writes its own lane and reads it, then @base. No token reads another PR's lane. Paths cannot name a lane (@ is outside the segment charset). contextful-gate passes base only from a default-branch push. A PR gate passes pr-<n>, and a dispatch that names no lane gets read-only. A caller that could name base already holds HMAC_SECRET, so the lane grants it nothing new.
  • CompileCacheLedger Durable Object, one per repository. Every PUT reserves its length against the repository bound and a new per-execution budget (COMPILE_CACHE_MAX_EXECUTION_BYTES, default 10 GiB). When the run ends, the workflow ends the execution's record, and the token can write nothing more. A deploy without the binding serves reads and returns 503 for every PUT.
  • Sweep. The sweep resumes from a cursor stored in R2 and holds one repository's listing at a time. For a repository listed whole within one tick, it evicts down to 90% and measures the size into the ledger. For a repository listed in part, it subtracts the bytes it deleted.
  • Eviction by last use. A GET of an entry older than half the TTL rewrites the entry after the response (waitUntil). The copy comes from R2 itself, so a PR token cannot change what the copy holds.
  • infra/overlay-limits.mjs carries the template's Durable Object bindings and migrations into an overlay whose migration tags are a strict prefix of the template's. An overlay whose migrations diverge is left unchanged.
  • Smaller fixes:
    • Only a run that armed the AgentBudget kills it at run end (run-end.ts). Before this, every compile-cache part made an AgentBudget RPC.
    • The command wrapper puts its closing ) on its own line, so a command that ends in a # comment still runs.
    • ADR 0007 and its index row describe the cache as built.

Key actions / todos

  • A PR now reuses entries only from its own earlier pushes and from @base. With the two-tier gate, @base holds only what the heavy parts compile, so light parts share no compiles across PRs. Follow-up: run the light parts' compiles on default-branch pushes, or measure the hit rate before deciding.
  • A PR's lane keeps its entries until the TTL, even after the PR closes. Deleting a lane when its PR closes is a possible follow-up.
  • ADR 0007 is still proposed.

lint 0, typecheck 0, pnpm test: 219 files, 3045 passed. wrangler deploy --dry-run lists the COMPILE_CACHE_LEDGER binding.

… at admission

A compile-cache token now carries a lane: base (default-branch executions),
pr-<n> (one pull request) or read-only. A pull request writes only its own
lane and reads it before base, so no pull request can plant an entry another
pull request or a default-branch gate links. contextful-gate names base only
from a default-branch push.

Every PUT reserves its length in a per-repository CompileCacheLedger Durable
Object against the repository bound and a per-execution budget; the workflow
ends the execution's record when its run finishes, which refuses that token's
further writes. A deploy without the binding serves reads and refuses writes.
The overlay merge carries the template's Durable Object bindings and
migrations into an overlay whose migrations lead the template's.

The sweep resumes from a stored cursor, holds one repository's listing at a
time, measures repositories it lists whole into the ledger and releases what
it deletes from the rest. A GET of an entry past half its TTL rewrites it, so
eviction and expiry count from last use.

Only an agent run that armed the AgentBudget kills it at run end, and the
command wrapper puts its closing parenthesis on its own line.

Spec: specs/adr/0007-cost-efficient-execution.md
@debuggingfuture
debuggingfuture marked this pull request as ready for review October 23, 2026 06:16
@debuggingfuture
debuggingfuture merged commit fa1fdb4 into main Oct 23, 2026
2 checks passed

@flaredispatch-fractalboxdev flaredispatch-fractalboxdev Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI code review — 🛑 Request changes

Risk tier: full · 1 critical · 1 warnings · 0 suggestions

Reviewers: security ⚠️ · performance ⚠️ · code-quality ⚠️ · documentation 0 · release-management 1 · compliance 1 · agents-md ⚠️

1. ⚠️ Warning — Token version bump invalidates active executions

📍 apps/dispatcher/src/compile-cache-token.ts:27

Changing the token version from cc1 to cc2 makes every token already issued by a prior deployment unverifiable, even though those tokens may remain valid until the execution ceiling. A rolling deployment can therefore cause in-flight workflows to lose compile-cache access or fail cache requests. Keep cc1 verification support during a compatibility window (while minting cc2), or explicitly drain/complete active executions before rollout.

2. 🛑 Critical — Compile-cache ledger binding is not provisioned by this change

📍 apps/dispatcher/src/env.ts:433-440

Declaring 'COMPILE_CACHE_LEDGER' in the 'Env' type does not create a Durable Object binding or migration in Wrangler configuration. The supplied diff adds no 'durable_objects.bindings' entry or migration to the deployment config, so production deployments will have this binding undefined and every compile-cache PUT will be refused with 503, despite the new ledger implementation. Add the binding and 'CompileCacheLedger' migration to the committed Wrangler configuration for each deployment.

📋 View full logs & reviewed diff ↗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant