Skip to content

A dry run marks a step unknown behind a cached prerequisite that restores outputs, though the restore gives it the exact files it recorded #938

Open

Description

@jasonkuhrt

Written by an AI agent (Claude) on behalf of @jasonkuhrt.

The dry-run RFC in #933 marks a step unknown when a step it waits on would run, or would be a hit that restores outputs. Its Future section names the next step, and the discussion on #933 moved it to an RFC of its own: if cache entries recorded each output's path and hash, a dry run could check those steps against the files the hit will restore. This is that ask.

What happens

The RFC's example on a fresh checkout: both builds cached from one successful run, nothing else changed, NODE_ENV included.

// vp run --dry-run --json -r build, as the RFC specifies it
"packages/ui#build": { /* … */ "steps": [{ /* … */ "cache": { /* … */ "status": "hit" } }] },
"apps/web#build": {
  /* … */
  "dependsOn": ["packages/ui#build"],
  "steps": [{ /* … */ "cache": { /* … */ "status": "unknown" } }]
}

The real run replays @acme/ui#build, which restores packages/ui/dist/index.js, and @acme/web#build then hits. The dry run can't say so: an entry's value holds its archive's file name but not what the archive holds (CacheEntryValue, crates/vt/src/session/cache/mod.rs:104-116 at c4b2aacf).

On a fresh checkout every generated file is absent until its producer runs or replays, so a cached producer is a hit that restores, and every step behind it that would otherwise be hit or changed reads unknown. A consumer that plans from the dry run, such as CI picking a pull request's jobs (vite-plus#2848), has to treat each of them as a step that may run.

The ask

Record what each entry's archive holds, and run today's check of a step that waits on hits against the files as those hits will leave them:

sequenceDiagram
    participant DR as vp run --dry-run
    participant UI as @acme/ui#35;build entry
    participant D as files on disk
    DR->>UI: check against the disk
    UI-->>DR: hit, will restore dist/index.js (hash h1)
    Note over DR: restored view = disk + what the hit restores
    DR->>D: read @acme/web#35;build's other recorded paths
    Note over DR: check @acme/web#35;build against the view:<br/>dist/index.js is h1 as recorded, the rest unchanged
    Note over DR: @acme/web#35;build is hit
Loading

What the check runs against

The restored view: the files as the hits a step waits on, directly or through other steps, will leave them.

  • Each archived path brings what the check compares for it: a file's hash, a symlink's target, or a directory.
  • The parent directories a restore creates are in the view too.
  • Every other path is as it is on disk.

The entry records what its archive holds when it archives its outputs.

What it checks

Today's check, unchanged, run against the view instead of the disk:

  • recorded files
  • missing paths
  • directories checked for existence
  • directory listings
  • cache.input globs, which can match a file only a hit restores

When a step stays unknown

  • It waits on a step that would run, or on one that is itself unknown.
  • It reads a path whose content depends on which of two unordered hits restores it last, and no hit ordered after both restores it again.

The RFC's other rules hold: only a step that would otherwise be hit or changed can be unknown, and noBaseline, error and disabled stay as they are.

Cost and compatibility

  • A remote hit costs no download. The dry run already reads the entry's value where a real run would, and the value records what the archive holds.
  • -vv: baseline.outputs gives each output with what the check compares for it, so a consumer can see what a hit will restore.
  • The cache schema version bumps, as it did for feat(cache): cache tasks that rewrite files they read #913, so existing caches fill again.

Our case

We'd like CI to pick a pull request's jobs from the dry run, checked against the entries our default branch uploads. We never commit generated source: in one local cache of our workspace, 104 of 601 step entries recorded at least one generated-source file, and each is absent on a fresh checkout until its producer replays. With this, those steps read as hit where the real run would hit.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions