Split the tests and cancel only the expensive job - #172
Merged
Merged
Conversation
`tests-and-type-check` became two jobs, because scoping cancellation usefully
first required separating the cheap work from the expensive work.
tests -m "not slurm" 3.13 and 3.14 ~1m
tests-slurm -m slurm 3.14 only ~6m
The two selections cover every test exactly once, verified by collection count:
26 + 21 = 47.
**Cancellation is scoped to `tests-slurm` alone.** It is the only job where
abandoning a superseded run saves anything worth having; the others finish in
under a minute and cancelling them would only make their status harder to read.
Only pull-request runs are abandoned, so a push to `main` keeps its run rather
than carrying a cancelled and therefore ambiguous status. A comment records the
trap for anyone who later gives this job a matrix: the matrix value has to go
into the group key, or the legs share one group and cancel each other.
**The duplicate mypy is gone.** `make check` in `quality` already runs it, so
the `Check typing` step was running the type checker a second and third time on
every matrix leg.
**The expensive job runs on one interpreter, deliberately.** It exercises the
scheduler, SSH and the container build rather than anything version-specific,
and running it twice cost more than the whole rest of CI put together. 3.14 is
what production runs. The fast job still covers both interpreters. This is a
coverage trade rather than a free win, so it is stated in the workflow and in
the plan rather than left implied.
Net runner time per push goes from roughly 13 minutes to roughly 9, and the
failure modes that fire most often now report in about a minute instead of
after six.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018TgVbCXXWAWhNjtKTzrxvN
jcschaff
force-pushed
the
ci/split-and-cancel-expensive-tests
branch
from
September 12, 2026 19:24
172b525 to
552bc3b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
concurrencywithcancel-in-progress, scoped as asked to the expensive tests only. Scoping it usefully first required splitting the cheap work from the expensive work, so that is here too.The split
tests-m "not slurm"tests-slurm-m slurmThe two selections cover every test exactly once. Verified by collection count rather than asserted: 26 + 21 = 47, against 47 collected overall.
Cancellation is on
tests-slurmaloneIt is the only job where abandoning a superseded run saves anything worth having. The others finish in under a minute, and cancelling them would only make their status harder to read.
Only pull-request runs are abandoned, so a push to
mainkeeps its run rather than carrying a cancelled and therefore ambiguous status.A comment records the trap for whoever later gives this job a matrix: the matrix value has to go into the group key, or every leg shares one group and the legs cancel each other.
Two things that fell out of the split
The duplicate mypy is gone.
make checkinqualityalready runs it, so theCheck typingstep was running the type checker a second and third time on every matrix leg.The expensive job runs on one interpreter, deliberately. It exercises the scheduler, SSH and the container build rather than anything version-specific, and running it twice cost more than the whole rest of CI put together. 3.14 is what production runs, per
.python-version. The fast job still covers both. This is a coverage trade rather than a free win, which is why it is stated in the workflow, in the plan document, and here rather than left implied — say so if you would rather keep the full matrix and pay the time.What it costs and saves
Combined with deleting the duplicate workflow in #171, runner time per push has gone from about 25 minutes to about 9.
🤖 Generated with Claude Code
https://claude.ai/code/session_018TgVbCXXWAWhNjtKTzrxvN