Skip to content

feat: upload each cache key once, default the sandbox transport to rpc, and bake a prebuilt DuckDB into the Rust image - #235

Merged
debuggingfuture merged 8 commits into
mainfrom
feat/cache-upload-once
Oct 22, 2026
Merged

debuggingfuture merged 8 commits into
mainfrom
feat/cache-upload-once

Conversation

@debuggingfuture

@debuggingfuture debuggingfuture commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Containers run on the rpc transport by default, the R2 cache uploads each content-addressed key once and skips the save on a restore it cannot apply, and the Rust sandbox variant ships a sha256-pinned prebuilt DuckDB shared library.

rpc transport by default

  • resolveSandboxTransport (packages/runtime-cf/src/sandbox-transport.ts) resolves one transport per execution: the CONFIG_KV pin sandbox.transport:<repo>, else the Worker-wide SANDBOX_TRANSPORT var, else rpc. A value that is not a transport name is skipped at its level.
  • The pin overrides the var in either direction, so an operator moves one repo back to http without a deploy.
  • makeCFRuntimeLive pins the resolved transport onto every container the execution acquires (sandbox-cf.ts, once per container) and hands the same value to the cache Layer. A container keeps the transport it started with for its life.
  • The dispatcher passes the raw pin and var through; the precedence lives in the one pure helper, covered by sandbox-transport.test.ts.
  • runs/README.md § Container transport documents the precedence, the rollback key and why rpc.

Upload each cache key once

  • RestoreFn reports hit, miss or unapplied. composeRestoreOr runs onMiss on miss and unapplied, and saves only after a miss. A restore error still degrades to a miss.
  • composeSave heads the key before packing. A stored archive skips tar czf and the upload, so parallel misses on one key upload once. A failed head check packs as before.
  • The live restore streams an archive in on rpc. On http / websocket it reads the key's head, reports unapplied without downloading the body, and logs the transport once per execution. An rpc apply that fails cancels the unread body and reports unapplied.
  • contextful-gate parts keep install: true, and a comment records why: the stage runs the consumer's contextful-ci, and the consumer's .cargo/config.toml can set net.offline, so nothing in this repo shows that the stage fetches its own crates.

Prebuilt DuckDB in the Rust sandbox variant

  • The WITH_RUST variant of infra/Dockerfile.sandbox-demo (RunSandbox) ships DuckDB's prebuilt libduckdb.so and headers under /opt/duckdb, so a consumer that links DuckDB skips the C++ compile that dominates a cold cargo test.
  • One ARG DUCKDB_VERSION pins the release; the libduckdb-linux-amd64.zip asset is verified with sha256sum -c against the digest GitHub publishes for it. Other architectures fail the build.
  • DUCKDB_LIB_DIR / DUCKDB_INCLUDE_DIR are exported, and an /etc/ld.so.conf.d/duckdb.conf entry puts the library on the loader path for every exec user. The build asserts ldconfig -p resolves libduckdb.so.
  • libduckdb-sys reads both variables only in its linked (non-bundled) build, and in a linked Linux build DUCKDB_LIB_DIR takes precedence over DUCKDB_DOWNLOAD_LIB=1. The consumer opt-in is documented on the layer and in the RunSandbox comment in wrangler.jsonc.
  • The layer ships in the WITH_RUST variant rather than the lean image the ADR names: RunSandbox is the only class with a Rust toolchain to build a linked consumer, and infra/Dockerfile.sandbox sets WITH_RUST for no class.

Spec: specs/adr/0007-cost-efficient-execution.md § Upload each cache key once, § Fan-out parts install nothing their stage repeats (the contextful-gate parts keep install: true, as above) and § Prebuilt heavy native dependencies in the image; specs/adr/0008-cheaper-waiting-and-transport.md § rpc transport by default (#232). ADR-0007 § Dispatch only work is #234.

Testing: pnpm lint, pnpm typecheck and pnpm test (207 files, 2836 passed, 1 skipped) pass. sandbox-transport.test.ts covers the default, the var, the pin in both directions and skipped values. cache-r2-live.test.ts runs the live cache closures against a mocked getSandbox and a fake bucket (http, websocket and rpc restores, an rpc miss, an rpc writeFile or tar xzf failure, a save of a stored key); cache-r2.test.ts covers unapplied and composeSave. Not run: an image build of the DuckDB layer and a linked cargo build against it; no PR CI builds images.

…lied restore

A restore reports hit, miss or unapplied. An unapplied archive runs the
miss path without a save; a save heads its key first and skips the pack
and upload when the archive is stored. On a transport that cannot stream
into the container, a restore reads only the head and logs the transport
once per execution.

Spec: specs/adr/0007-cost-efficient-execution.md (#232)
…describe the unapplied outcome

The live cache tests cover an rpc restore with no stored archive (miss, pack,
upload), an rpc restore whose tar xzf exits non-zero (unapplied, no save) and a
websocket restore (head only). Comments and runs/README.md describe a restore
off the rpc transport as unapplied rather than a miss, and the transport log as
once per Layer instance.

Spec: specs/adr/0007-cost-efficient-execution.md (#232)

@flaredispatch-fractalboxdev flaredispatch-fractalboxdev Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI code review — 💬 Comment

Risk tier: full · 0 critical · 1 warnings · 0 suggestions

Reviewers: security ⚠️ · performance 1 · code-quality 1 · documentation 0 · release-management ⚠️ · compliance ⚠️ · agents-md ⚠️

1. ⚠️ Warning — Concurrent cache misses still duplicate packing and uploads

📍 packages/runtime-cf/src/cache-restore-or.ts:48-63

'composeSave' performs a non-atomic 'exists' check followed by 'pack'. Parallel executions can all observe a missing key, then each run the expensive tar creation and upload, contradicting the stated upload-once behavior and potentially causing substantial duplicate CPU, I/O, and R2 traffic. Add per-key coordination/locking or use an atomic create-if-absent mechanism before packing/uploading.

📋 View full logs & reviewed diff ↗

…olved transport onto every container

resolveSandboxTransport takes the CONFIG_KV sandbox.transport:<repo> pin, else the Worker-wide SANDBOX_TRANSPORT var, else rpc, skipping a value that is not a transport name at either level. The runtime pins the result onto every container it acquires and hands it to the R2 cache, so a restore streams an archive in unless a repo or the Worker opts back to http.
@debuggingfuture debuggingfuture changed the title feat(cache): upload each cache key once and skip the save on an unapplied restore feat: upload each cache key once, default the sandbox transport to rpc, and bake a prebuilt DuckDB into the Rust image Oct 22, 2026

@flaredispatch-fractalboxdev flaredispatch-fractalboxdev Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI code review — 🛑 Request changes

Risk tier: full · 1 critical · 2 warnings · 1 suggestions

Reviewers: security 0 · performance 2 · code-quality 0 · documentation 1 · release-management 0 · compliance 2 · agents-md 0

1. ⚠️ Warning — Every pnpm cache restore performs another install

📍 packages/core/src/primitives/install-cached.ts:154-163

The new 'relink' command runs after every restore, including cache hits and cache misses. 'pnpm install --frozen-lockfile --prefer-offline' still performs dependency resolution and filesystem work, and on a miss it repeats the install that 'onMiss' already performed. This adds avoidable latency to every pnpm run and can negate the cache benefit; use a narrowly scoped relink operation or only run it on hits when the archived tree actually requires workspace links.

2. ⚠️ Warning — Concurrent misses can still pack and upload the same cache key

📍 packages/runtime-cf/src/cache-restore-or.ts:51-63

The 'exists' check and 'pack' call are separate, non-atomic operations. Parallel executions can both observe a missing key, independently run the expensive archive creation, and upload duplicate archives, despite the comment promising upload-once behavior. Add per-key coordination or an atomic/conditional creation mechanism before packing, or accept and document the duplicate-work behavior.

3. 💡 Suggestion — Clarify that rpc is only the default transport

📍 runs/README.md:505-506

The opening sentence says containers run on the SDK's 'rpc' transport, but the documented configuration can resolve them to 'http' or 'websocket' via the repo pin or 'SANDBOX_TRANSPORT'. Reword this to say containers default to 'rpc' and use the resolved transport according to the precedence below, so readers are not given an unconditional claim that contradicts the configuration instructions.

4. 🛑 Critical — DuckDB license files are omitted from the distributed image

📍 infra/Dockerfile.sandbox-demo:171-172

The image copies only 'libduckdb.so' and three headers from the DuckDB release archive; no DuckDB license or copyright notice is retained. Since this adds and distributes DuckDB binaries/headers, preserve the applicable license notices in the image or another distributed compliance location.

📋 View full logs & reviewed diff ↗

@debuggingfuture
debuggingfuture merged commit 724ca24 into main Oct 22, 2026
5 checks passed
@debuggingfuture
debuggingfuture deleted the feat/cache-upload-once branch October 22, 2026 11:08

@flaredispatch-fractalboxdev flaredispatch-fractalboxdev Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI code review — 💬 Comment

Risk tier: full · 0 critical · 2 warnings · 0 suggestions

Reviewers: security ⚠️ · performance 2 · code-quality 1 · documentation ⚠️ · release-management ⚠️ · compliance ⚠️ · agents-md 0

1. ⚠️ Warning — Concurrent cache misses still duplicate packing and uploads

📍 packages/runtime-cf/src/cache-restore-or.ts:52-64

The existence check and pack are not atomic: concurrent executions can all observe 'exists=false', then each run 'pack(opts)' and upload the same content. This defeats the stated upload-once optimization and can multiply tar/stream/upload CPU, container time, and R2 bandwidth under fan-out. Add per-key coordination or an atomic conditional-write/claim mechanism before packing.

2. ⚠️ Warning — Every cached install now starts a second pnpm install

📍 packages/core/src/primitives/install-cached.ts:156-164

The relink command runs unconditionally after 'restoreOr', including cache misses where 'onMiss' has already performed the full pnpm install and populated workspace links. This adds a pnpm startup and workspace/lockfile scan to every miss, and a potentially expensive install check to every hit. Restrict relinking to successful cache restores, or use a purpose-built link-only operation that cannot re-resolve or reinstall dependencies.

📋 View full logs & reviewed diff ↗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant