feat(vllm-cpp): enable and vendor the MLX GEMM provider on darwin/metal

[localai-bot]

What

Builds the darwin vllm-cpp image with vllm.cpp's optional MLX GEMM provider and vendors its runtime into the backend package.

vllm.cpp keeps VLLM_CPP_MLX OFF by default because it costs a ~19 MB libmlx.dylib plus a ~105 MB mlx.metallib, on the explicit position that it must earn that cost by measurement.

Why: it earns it

Measured on an Apple M4 (16 GiB, macOS 26.5.2). One binary, arms toggled with VT_OP_PROVIDER_DISABLE=mlx so there is no build-difference confound. Qwen3-1.7B-bf16, p=512 g=128, 2 reps, arm order alternated per rep to cancel thermal drift.

B	MLX agg tok/s	native agg tok/s	speedup	MLX TTFT	native TTFT
1	5.79	3.08	1.88x	3.32 s	7.68 s
2	10.19	6.08	1.68x	5.09 s	11.68 s
4	15.75	10.24	1.54x	9.63 s	18.77 s
8	25.70	13.69	1.88x	13.95 s	34.38 s
16	38.65	17.69	2.19x	18.33 s	54.48 s

Peak RSS is unchanged (6.65 to 7.50 GB in both arms), and the output is bit-identical: vllm.cpp's three-way parity test reports mlx-vs-msl NMSE of 0 on all six shapes and mlx-vs-cpu equal to msl-vs-cpu, against a 5e-4 bar.

MLX serves the dense GEMM only. Paged attention stays vllm.cpp's own kernel, because MLX has no paged-KV primitive at all.

Honest limits: the numbers are INDICATIVE, not binding. The three actions.runner agents on the box were verified idle, booted out for the sweep and restored, and the whole A/B ran inside one lockf so nothing could interleave between arms, but com.localai.worker and the aerial wallpaper stayed up (no passwordless sudo). Rep spread reached 9.4% on 2 reps, so read the multipliers as +/-10%; the gap far exceeds the noise. Full disposition in vllm.cpp docs/BENCHMARKS.md, "MLX GEMM provider A/B on Apple M4" (mudler/vllm.cpp@41d7f8d7).

How

Build. MLX comes from the pinned prebuilt pip wheel (MLX_VERSION, default 0.29.3) into a venv under the backend dir. Building MLX from source needs xcrun metal, a full Xcode the macOS runners do not have; the wheel ships include/, lib/libmlx.dylib and the compiled metallib ready to link. The install is a stamp file rather than a phony target, since a phony prerequisite is always newer than libvllm and would re-link it on every invocation. VLLM_CPP_MLX=off restores the previous Metal build.

Packaging. Vendors libmlx.dylib, mlx.metallib and MLX's MIT license into package/lib/. Three things this had to get right, each verified on the M4 before it was written:

1. libvllm.dylib links @rpath/libmlx.dylib and its build-time LC_RPATH points inside the build venv, a path no user has. Every build rpath is deleted and replaced with @loader_path/lib.
2. MLX loads its metallib from beside its own dylib, so both files must land in the same directory or every Metal op fails with "Failed to load the default metallib".
3. install_name_tool invalidates the code signature and macOS refuses to load an arm64 image with a stale one, so the patched library is re-signed ad-hoc.

Verification

Built through this Makefile on the M4 and ran the packaged artifact:

• DYLD_PRINT_LIBRARIES resolves libmlx from package/lib/libmlx.dylib, not the build venv
• zero occurrences of the build-venv path left in the load commands
• codesign -v passes on the patched library
• a real generation runs with the provider selected (op=65 selected=mlx priority=100) and zero metallib failures

A missing rpath now fails the build rather than the user's first inference request.

Also caught and fixed during verification: make expands a whole recipe before running its first line, so the original $(shell) glob for MLX_ROOT resolved before the venv existed. It now resolves in the shell inside the stamp recipe.

Cost

The darwin vllm-cpp image grows by about 124 MB.

Notes

• No Go files change; Linux and CUDA/Vulkan builds are untouched.
• The darwin matrix entry already exists (-metal-darwin-arm64-vllm-cpp), so no CI matrix change is needed.
• Possible follow-up: wire MLX_VERSION into the existing dep-bump automation, as done for vllm-metal.