Skip to content

feat(cache): warm every contextful compile group into the base lane - #242

Open
debuggingfuture wants to merge 2 commits into
mainfrom
cache-warm-groups
Open

debuggingfuture wants to merge 2 commits into
mainfrom
cache-warm-groups

Conversation

@debuggingfuture

@debuggingfuture debuggingfuture commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

contextful-cache-warm warms every distinct compile of the contextful gate into the base lane, not only workspace.compile.

Problem & Insight

  • The gate's CLI shards, CLI differential part, package groups, evaluate, features and acceptance parts each resolve their own cargo invocation and feature set. A cache filled by cargo test --workspace --no-run misses all of their crate compiles.
  • contextful's contextful-ci names those compiles itself. stages --parts --compile-groups lists one part per distinct set of cargo test invocations, and gate --stage <part> --compile-only runs that part's invocations with --no-run. Warming through these two commands keeps the sccache keys identical to the real parts' keys.

Take

  • The parent checks out the head and builds the prebuilt contextful-ci. It then lists the compile groups for each tier and the folds.
  • The groups spread round-robin over at most WARM_CHILDREN (4) children of the same run. Each child compiles its share in order in one container, under the gate part's command, checkout, install and cache wrapper, plus --compile-only. A child starts no group past 2 h, and a failed or timed-out group does not stop the next one. A tree of the current size, with 31 groups, does not fit one container's deadline, so the groups are split across children.
  • The parent releases its container and admission slot, joins the children and reports each child's groups and status.
  • A tree whose contextful-ci lacks the flag, or a head with one group, compiles in the parent's container. With the flag missing, the part is workspace.compile, so behavior matches feat(cache): warm the compile cache's base lane from every default-branch head #241.
  • feat(cache): warm the compile cache's base lane from every default-branch head #241's boundaries stay in place:
    • The parent and every child re-read the branch head and skip a stale one.
    • The supersession group and the idempotency key are unchanged.
    • Children join no supersession group, because superseding the parent cancels its tree (joinsChildren).
    • No part check-runs are posted.
  • With gate.lightLabels set, the warm leaves out each pull-request group whose own label and fold label are both heavy, because the heavy tier compiles it on the same push. Nightly groups always warm.
  • stageCommand takes { compileOnly }. ciCommand is exported for the discovery commands.

Spec: specs/adr/0007-cost-efficient-execution.md § Base-lane warm.

Verification

  • pnpm typecheck, pnpm lint: clean. oxlint@1.79.0, the version the flare-dispatch/oxlint check pins, is clean once run-end.test.ts types its end mock instead of declaring an unused _ parameter.
  • vitest runs/contextful-cache-warm.test.ts: 13 passed. The tests cover the fallback, the child spread and join, the order of a child's compile-only commands past a failing group, the light-label filter, and the skips.
  • pnpm test: 3057 passed. One failure, check-command-supervisor "TERM-ignoring descendant", is timing-sensitive under full-suite load and passes when run alone (13/13).

contextful-cache-warm lists the head's compile groups with
`stages --parts --compile-groups` per tier and compiles each through
`gate --stage <part> --compile-only` under the gate part's command,
checkout, install and cache wrapper. The groups spread round-robin over
at most four children of the run, each compiling its share in one
container and starting no group past 2 h; the parent hands its container
and slot to them and joins. A tree without the flag compiles
workspace.compile in the parent, as before. With gate.lightLabels set,
heavy pull-request groups are left to the heavy tier. Children re-check
the branch head and join no supersession group; the parent's carries
them.

Spec: specs/adr/0007-cost-efficient-execution.md

@flaredispatch-fractalboxdev flaredispatch-fractalboxdev Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI code review — 💬 Comment

Risk tier: full · 0 critical · 2 warnings · 0 suggestions

Reviewers: security ⚠️ · performance 1 · code-quality 0 · documentation 1 · release-management ⚠️ · compliance ⚠️ · agents-md ⚠️

1. ⚠️ Warning — Deduplicate compile groups before scheduling

📍 runs/contextful-cache-warm.ts:96-102

'warmableGroups' concatenates the PR and nightly listings without deduplicating them. If a group appears in both tier outputs (or is repeated by the CLI), it is compiled more than once, wasting child capacity and cache-warming time. Deduplicate while preserving run order before chunking and spawning children.

2. ⚠️ Warning — Clarify the single-group behavior

📍 specs/adr/0007-cost-efficient-execution.md:118-120

This sentence implies that a head with one compile group also compiles 'workspace.compile', but the implementation compiles the one listed group (which may be 'features.binary-all' or another group) in the parent's container. Restrict the 'workspace.compile' claim to the no-groups fallback and separately state that a single listed group is compiled directly.

📋 View full logs & reviewed diff ↗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant