[debuggingfuture]

Every contextful gate part compiles the same workspace cold. This PR gives those parts a shared sccache cache backed by the dispatcher's existing R2 binding, so after the first part compiles a crate, later parts read it from the cache instead of compiling it again.

Problem & Insight

• About 30 parts run per gate, and each one recompiles the workspace from scratch: workspace.compile takes 1642 s and the CLI shards take 18–40 min each. Most of that work is the same rustc invocations repeated with identical inputs.
• sccache's WebDAV backend (OpenDAL services-webdav, sccache 0.18) uses only GET, PUT, Depth: 0 PROPFIND and MKCOL. Before each PUT it runs a PROPFIND on the parent. If the server reports every collection as existing, a write costs one PROPFIND plus one PUT, and MKCOL is never sent.
• If sccache's storage check fails, the sccache server refuses to start, and a compile that needs that server fails. SCCACHE_IGNORE_SERVER_IO_ERROR does not help here because it only applies once the server is running. So starting the server has to be the explicit fallback point.
• Check commands are fingerprinted over their env, so the token must come out identical on every replay of the workflow.

Take

• Route (routes/compile-cache.ts): serves /v1/compile-cache/<owner>/<name>/… from RUNS_STORAGE under compile-cache/<owner>/<name>/. It is wired in before the typed router (an escape hatch) so PUT bodies stream straight into R2.
  • Bearer token handling: no token → 401. A token for another repo, a forged token or an expired one → 403.
  • Size limits: an entry over COMPILE_CACHE_MAX_OBJECT_BYTES → 413. A write that would push the repo's prefix past COMPILE_CACHE_MAX_REPO_BYTES (as measured by the last sweep) → 507.
  • An hourly sweep (41 * * * *) deletes entries older than COMPILE_CACHE_TTL_DAYS. If a repo is still over its ceiling, it deletes the oldest-written entries until the repo is at 90% of the ceiling.
  • The limits are wrangler vars, with defaults in code: 100 MiB per object, 20 GiB per repo, 14 days.
• Token (compile-cache-token.ts):
  • It is an HMAC under a key derived (HKDF) from HMAC_SECRET, so no new secret is needed.
  • It is bound to the repo and the execution. The repo comes from the request path.
  • It expires at instance creation + queue age + maxDurationSec. Because it is derived from the creation time, every invocation mints the same token.
• Runs:
  • RunSpec.compileCache lets a run opt in. When it does, the workflow injects the endpoint and token through config overrides.
  • check reads them directly (inline, not in a checkpointed step), wraps the command with compileCacheCommand and redacts the token from logs.
  • contextful-gate sets compileCache: true on every part.
• Wrapper:
  • It computes SCCACHE_WEBDAV_KEY_PREFIX=<repo>/<rustc release>/<host triple> from rustc -vV inside the checkout.
  • It exports CARGO_INCREMENTAL=0, SCCACHE_IDLE_TIMEOUT=0 and SCCACHE_IGNORE_SERVER_IO_ERROR=1.
  • It runs sccache --start-server first and only sets RUSTC_WRAPPER if the server starts.
  • At the end it prints sccache --show-stats into the part's log and exits with the command's own status.
• Image: the Rust variant of Dockerfile.sandbox-demo installs sccache v0.18.0 (static musl build), checked against a pinned SHA-256.

Verified locally with sccache 0.18.0 against this route served over Node HTTP:

• The first build misses and writes the entry. A second build with a fresh target dir reports Cache hits 1, Cache hits rate 100.00 %.
• With a bad token, the server does not start, the build compiles locally and exits 0.

Key actions / todos

• Tests:
  • token scope (missing, expired, wrong repo, rewritten expiry, malformed), 503 with no secret
  • WebDAV round trip and key prefixing; Depth and method refusals; 411/413/507 limits; path traversal
  • sweep TTL, eviction and usage records; scheduled routing; cron parity
  • wrapper fallback, statistics and exit status under /bin/sh with stub binaries
  • check env wiring and token redaction; gate parts request the cache
• Measure per-part hit rates from the --show-stats lines on the next gates, then decide whether the 20 GiB ceiling holds the workspace's feature sets.
• Trust boundary: pull-request parts from the same repository write to the shared prefix, so their entries can be read by later default-branch parts. If that is not acceptable, splitting into a PR-writable prefix and a default-branch read-only prefix is a follow-up.

[flaredispatch-fractalboxdev]

AI code review — 🛑 Request changes

Risk tier: full · 1 critical · 0 warnings · 0 suggestions

Reviewers: security ⚠️ · performance 1 · code-quality ⚠️ · documentation ⚠️ · release-management ⚠️ · compliance 1 · agents-md 0

1. 🛑 Critical — Truncated sweeps can undercount repository usage and bypass the byte ceiling

📍 apps/dispatcher/src/routes/compile-cache.ts:278-321

When the R2 listing reaches MAX_SWEEP_PAGES, the sweep marks the result truncated but still records usage metadata for repositories present in the partial listing. PUT admission then trusts that lower-bound 'bytes' value, so a large repository can be admitted as if it were only partially measured and can exceed COMPILE_CACHE_MAX_REPO_BYTES without bound. Do not persist or use partial measurements for admission; retain the prior usage record or fail closed until a complete measurement succeeds.

📋 View full logs & reviewed diff ↗