Repository navigation
CI: the nightly's macOS leg runs on a self-hosted Apple silicon runner - #19
Closed
IsuraManchanayake wants to merge 12 commits into
Closed
IsuraManchanayake wants to merge 12 commits into
IsuraManchanayake wants to merge 12 commits into
Conversation
…cords the result Each adapter installs on its own only the upstream versions it was built and tested against, so those lists go stale as llama.cpp and MLX release. The job picks the newest upstream release the recipe does not list (scripts/nightly_runtimes.py), installs it untested, and runs the adapter's tests on it: llama.cpp on macos-14 and ubuntu-24.04 with a small GGUF for the model tests, MLX on macos-14 with the worker built as a release ships it. A llama.cpp build that passes arrives as a PR adding it to the recipe with the size and SHA-256 GitHub lists for each of its assets, the same asset names the previous tested build carries; one that fails opens an issue with the run attached. An MLX version is pinned by hand, wheel by wheel, so its result is an issue either way. workflow_dispatch tests a named release instead of the newest.
…Python recipe::install leaves two site-packages: the bundled interpreter's under python/ and the environment's under env/, where mlx and mlx-lm went. The job took the first find found, which can be the bare interpreter's, and built its test executables against whatever Python PyO3 picked on the runner. Now SUPERFLUID_MLX_VENV names env/, checked by importing mlx and mlx-lm from it, and PYO3_PYTHON names the install's Python 3.12.
An MLX version that passes the adapter's tests arrives the same way a llama.cpp build does: as a PR adding it to the adapter's recipe, built from pip's report of the environment it resolved on the runner, every wheel with the size and SHA-256 PyPI lists for it. The issue stays for a version that fails, which includes one needing a newer Python than the adapter ships.
A dry-run resolution made after the install could name other versions or wheels than the environment installed moments before, and the recipe would pin those as tested. The report now comes from a real install into the test environment, which the tests then run on.
- A listing that fails (issues disabled, no permission) was read as an open issue and the run recorded nothing: every listing check now stops with a clear error. - The MLX test executable, built against the install's Python, links @rpath/libpython with no rpath into the install: the install's lib directory is named for dyld. - Hosted runners share GitHub's unauthenticated API rate limit, and the worker's plan of an untested build hit it: the kit sends GITHUB_TOKEN for api.github.com when set, and the plan step has the job's token. - A runtime has one open issue; a later build that fails too is noted on it instead of opening one a day. Seen on the fork: https://github.com/IsuraManchanayake/superfluid/actions/runs/38715098729
…e adapter runs on pip on a runner or machine newer than macOS 14 resolves the mlx and mlx-metal wheels built for its own macOS, which an older Mac cannot install. The recording script now pins, for a wheel with a macOS tag, the same wheel built for the oldest macOS at or above 14 that PyPI offers, with the size and digest PyPI lists for it.
The hosted macos-14 runner's GPU is virtualized: llama.cpp takes its fallback Metal kernels there, a 40-token prompt costs seconds, a padded batch answers a zero logit row, and a batched round differs from solo decode past the tests' bound, whatever the build; and with 2 GB free the adapter keeps one shared pool where the tests expect a buffer per lane. The leg now asks for a self-hosted Apple silicon runner and keeps the toolchain that runner has.
The runner's PATH is its owner's shell's at start, which need not name ~/.cargo/bin; the first run on a Mac died in fifteen seconds on 'cargo: command not found'.
…stup Swatinem/rust-cache's save step removes every regular file in ~/.cargo/bin that cargo did not install during the job: on a hosted runner that is nothing of value, on a developer's Mac it was rustup itself. A self-hosted runner keeps its work directory between jobs and needs no cache.
…e file Its key is the workflow file's hash, so every edit to the workflow missed the cache and the post step uploaded the model from the runner: on a laptop, minutes at a megabyte a second.
Collaborator
Author
|
Folded into #13 to keep the PR count down: the same five commits are appended there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #13: its diff includes that PR until it merges; this PR's own commits are the last five. I will rebase it onto
rc-0.1.3as it lands.What changed
The nightly's llama.cpp macOS leg asks for a self-hosted Apple silicon runner (
[self-hosted, macOS, ARM64]) instead of the hostedmacos-14. On a self-hosted runner the job keeps the toolchain the machine has (rustup's bin directory on the job's PATH, the toolchain action skipped), runs no rust-cache, and uses no model cache: the machine keeps its work directory and the model between jobs. The recipe PR's text names where the tests ran.Why
The hosted macos-14 runner cannot pass the adapter's real-model tests on any build. Its virtualized GPU takes llama.cpp's fallback Metal kernels: a 40-token prompt costs seconds, a batched round differs from solo decode past the tests' bound, and a padded batch answers a zero logit row. Its 7 GB leaves the adapter one shared pool where the tests expect a buffer per lane. The failures were bit-identical across two runs (https://github.com/IsuraManchanayake/superfluid/actions/runs/38725440780, https://github.com/IsuraManchanayake/superfluid/actions/runs/38739018175), and the same suite passes on real Apple silicon, which is what a tested build has to mean for a Mac.
rust-cache is excluded on self-hosted runners because its save step deletes every regular file in
~/.cargo/binthat cargo did not install during the job; on a developer's Mac that was rustup itself (https://github.com/IsuraManchanayake/superfluid/actions/runs/38744781241/job/116457149024). The model cache is excluded because its key is the workflow file's hash, so every edit re-uploaded 386 MB from the runner.How it was tested
main, both legs passed on llama.cpp b11658 (85 adapter tests on the Mac, the layout read as V2), and the job opened llamacpp: tested with b11658 IsuraManchanayake/superfluid#15 with the recipe entry, 15 assets with the digests GitHub lists.runblock passesbash -n.Reviewed in the fork as IsuraManchanayake#14.
cargo clippy --all-targets -- -D warningscargo test