Skip to content

CI: opt-in DevLab ephemeral runner lanes for Strix Halo (Linux + Windows) - #297

Draft
rradjabi wants to merge 1 commit into
ROCm:mainfrom
rradjabi:devlab/ephemeral-runner-lanes
Draft

CI: opt-in DevLab ephemeral runner lanes for Strix Halo (Linux + Windows)#297
rradjabi wants to merge 1 commit into
ROCm:mainfrom
rradjabi:devlab/ephemeral-runner-lanes

Conversation

@rradjabi

@rradjabi rradjabi commented Aug 21, 2026

Copy link
Copy Markdown

What this does

Adds two dispatch-only jobs to e2e-selfhosted.yml that run the existing E2E suite on the AMD Ryzen DevLab ephemeral pool, alongside (not instead of) the current strix-halo-ubuntu / strix-halo-windows persistent runners:

E2E tests (Strix Halo, Ubuntu, ephemeral) runs-on: [self-hosted, linux, strix-halo, ephemeral]
E2E tests (Strix Halo, Windows, ephemeral) runs-on: [self-hosted, windows, strix-halo, ephemeral]

On the ephemeral pool a runner is registered for one job and destroyed with its working directory afterwards.

Merging this is a no-op

Both lanes are gated on github.event_name == 'workflow_dispatch' with an exact platform match, and all deliberately does not include them. On push / PR / merge_group they skip. The diff is +292 / −1, and the single deleted line is the platform options list being reformatted to add two entries — no existing job is modified.

You run them when you choose to:

gh workflow run e2e-selfhosted.yml --ref <branch> -f platform=strix-ubuntu-ephemeral
gh workflow run e2e-selfhosted.yml --ref <branch> -f platform=strix-windows-ephemeral

Prerequisite: install the App first

Nothing provisions a runner until the DevLab orchestrator GitHub App is installed on this repo:

Why this isn't a label swap

The three differences from the native lanes are forced by the target hosts, and each one would be a real bug if copied across:

  1. No host-path pinning. The native Ubuntu lane pins HOME, CARGO_HOME, RUSTUP_HOME, TMPDIR, PIP_CACHE_DIR under /home/ubuntu/actions-runner because everything else on that box is on a full root partition. The pool's hosts keep / on a tmpfs overlay and set TMPDIR, PIP_CACHE_DIR, XDG_CACHE_HOME, HF_HOME and the ROCm/MIOpen cache vars themselves, pointed at their own persistent nvme — and deliberately leave HOME alone. Carrying the hardcoded paths over would put job scratch in RAM on a machine whose iGPU takes its VRAM from the same 62 GiB.

  2. No GPU reclaim step. pkill-ing serves leaked by a prior job is meaningless on a runner that has never run one. This is the property the migration actually buys.

  3. The install sdk pre-warm moves off RUNNER_WORKSPACE. The native lanes note it "persists across runs on RUNNER_WORKSPACE" — on an ephemeral runner it does not; the work tree is wiped with the runner. It's pointed at the pool's persistent cache root instead (XDG_CACHE_HOME on Linux, LOCALAPPDATA on Windows), which survives both the job and a reboot, so the pre-warm stays a once-per-host cost. Without this every run re-pays it. This is the change most worth reviewing — it's the one that decides whether ephemeral is competitive on wall-clock.

CARGO_TARGET_DIR intentionally stays on the ephemeral work tree: build output is per-commit, so persisting it buys little and costs disk the pool would have to reclaim. Expect the first ephemeral run on each host to be slow.

Routing is unambiguous

The pool's hosts carry strix-halo + ephemeral and never native; your persistent runners carry native and never ephemeral. Neither pool can claim the other's jobs. [self-hosted, linux] alone would not reach the pool either — it refuses any job naming only generic labels, specifically so it never picks up untargeted work.

Reporting

e2e-report already globs *-report, so the two new artifacts appear as new platforms with no change there. Both lanes are added to its needs so an ephemeral trial produces a consolidated report you can diff against the native one — that comparison is the point of the trial.

Not included

Removing the native lanes, and any branch-protection / required-check changes. Those belong after a trial, not before it.

Adds two dispatch-only jobs alongside the existing native Strix lanes,
targeting the AMD Ryzen DevLab ephemeral pool: a runner is registered
for one job and destroyed with its working directory afterwards.

Nothing changes on push/PR/merge_group. The new lanes are gated on
workflow_dispatch with an exact platform match, and `all` deliberately
does not include them, so merging this cannot affect any existing lane.

The ephemeral lanes are not a label swap of the native ones. Three
differences are forced by the target hosts:

  - No host-path pinning. The native Ubuntu lane pins HOME, CARGO_HOME,
    RUSTUP_HOME, TMPDIR and PIP_CACHE_DIR under /home/ubuntu/actions-runner
    because everything else on that box is on a full root partition.
    The pool's hosts keep / on a tmpfs overlay and set those variables
    themselves, pointed at their own persistent nvme, so inheriting is
    correct and copying the paths across would put job scratch in RAM on
    a machine whose iGPU takes its VRAM from the same 62 GiB.

  - No GPU reclaim step. pkill-ing serves leaked by a prior job is
    meaningless on a runner that has never run one.

  - The `install sdk` pre-warm moves off RUNNER_WORKSPACE, which does not
    survive an ephemeral runner, onto the pool's persistent cache root.
    Without this every run re-pays the pre-warm.

Requires the DevLab orchestrator GitHub App to be installed on this
repository before either lane is dispatched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant