CI: a nightly job tests each runtime's newest upstream release and records the result

[IsuraManchanayake]

What changed

A scheduled workflow installs the newest upstream llama.cpp build and MLX version untested, through each adapter's own plan --untested and install, and runs the adapter's tests on it: llama.cpp on macos-14 and ubuntu-24.04 with a small GGUF, MLX on macos-14 in the install's own environment and against its own Python. A release that passes arrives as a PR adding it to the adapter's recipe (scripts/nightly_runtimes.py <runtime> append): for llama.cpp with the size and SHA-256 GitHub lists for each asset, for MLX with the environment pip installed for the tests, every wheel with the size and SHA-256 PyPI lists for it, read from a real pip install --report. One that fails opens an issue with the run attached. workflow_dispatch tests a named release.

$ python3 scripts/nightly_runtimes.py llamacpp pick
version=b11630
$ python3 scripts/nightly_runtimes.py llamacpp append b11630
note: b11630 lacks llama-b11630-bin-ubuntu-openvino-x64.tar.gz; left out
recorded b11630 (2026-10-14) with 15 assets in crates/superfluid-adapters/superfluid-adapter-llamacpp/recipe.json

A schedule fires only from the default branch, so the job needs merging to main and the Actions setting that lets workflows open pull requests.

The llama.cpp macOS leg runs on a self-hosted Apple silicon runner ([self-hosted, macOS, ARM64]), not the hosted macos-14. On a self-hosted runner the job keeps the machine's toolchain (rustup's bin directory on the job's PATH, the toolchain action skipped), runs no rust-cache and uses no model cache, since the machine keeps its work directory and the model between jobs.

Why

The tested lists in the recipes are what superfluid compat declares and what a release installs on its own; without a job they fall behind upstream, and a human types digests. See the new section "How an upstream release becomes a tested one" in docs/models/runtimes.md.

The hosted macos-14 runner cannot pass the adapter's real-model tests on any build: its virtualized GPU takes llama.cpp's fallback Metal kernels, where a batched round differs from solo decode past the tests' bound and a padded batch answers a zero logit row, and its 7 GB leaves the adapter one shared pool where the tests expect a buffer per lane. The failures were bit-identical across runs (https://github.com/IsuraManchanayake/superfluid/actions/runs/38725440780, https://github.com/IsuraManchanayake/superfluid/actions/runs/38739018175); the same suite passes on real Apple silicon, which is what a tested build has to mean for a Mac. rust-cache stays on hosted runners because its save step deletes every regular file in ~/.cargo/bin that cargo did not install during the job, rustup included on a developer's Mac (https://github.com/IsuraManchanayake/superfluid/actions/runs/38744781241/job/116457149024); the model cache too, since its key is the workflow file's hash and every edit re-uploaded 386 MB from the runner.

How it was tested

• pick and append run locally against GitHub and PyPI: b11630 and 0.32.4 picked; a llama.cpp entry appended with 15 of 16 assets; an MLX entry rebuilt from a pip report of the recipe's own 0.32.2 environment reproduces the recipe byte for byte; a digest PyPI does not list, a report that resolved another mlx, a missing report and an already tested version are refused with the recipe untouched.
• The workflow parses as YAML and every run block passes bash -n.
• Run on the fork from its main, five dispatches. The first three found a listing failure read as "already open", the MLX test binary's missing libpython rpath and GitHub's unauthenticated rate limit on hosted runners, all fixed here. It then found llama.cpp b11642's new context-parameter layout (Llamacpp: the b11642 layout of llama_context_params beside the older one, and exports in host memory on a CPU-only host #17) and MLX 0.32's cache API (Mlx: the shim drives mlx-lm 0.32's caches, whose state carries everything, beside 0.31's #18), both real. With a MacBook Pro as the self-hosted runner it recorded b11658 end to end: https://github.com/IsuraManchanayake/superfluid/actions/runs/38751331332 opened llamacpp: tested with b11658 IsuraManchanayake/superfluid#15 with the entry.
• Not run: a lab Mac as the runner; the org's runner group has to admit this repository first. A leg with no matching runner online waits in the queue and holds later runs behind it through the workflow's concurrency group.

Reviewed in the fork as IsuraManchanayake#4, #5 and #14.

• cargo clippy --all-targets -- -D warnings
• cargo test
• Changes a wire protocol, the WAL format or the tick ABI (says so above, version bumped)