Two launch-path bugs found benchmarking: served_model_name routing break, silent instance replace

[webdevtodayjason]

Both of these came out of running config A/B tests on spark-3. Filing rather than fixing, because the fixes are design calls I'd rather not make unilaterally.

served_model_name breaks federated routing

A model loaded with served_model_name gets advertised fleet-wide under its repo id, while the engine only answers to the alias. So the fleet menu lists a model that 404s on every request.

POST /api/models/load {"model":"unsloth/Qwen3.8-27B-NVFP4","served_model_name":["qwen38-ctl"]}

proxy  /v1/models   ->  unsloth/Qwen3.8-27B-NVFP4
engine /v1/models   ->  qwen38-ctl

POST proxy   model=unsloth/Qwen3.8-27B-NVFP4  ->  404 "The model does not exist."
POST engine  model=qwen38-ctl                 ->  200

We already track phantom menu entries, but those need a dead engine. This one needs a healthy engine and a documented API field, which makes it a lot easier to hit. It also means served_model_name can't do the thing it exists for, since the short name is precisely what the router can't resolve.

My preference is announcing the served names alongside the repo id so the router resolves both, rather than having the proxy rewrite model on the way out. That keeps the fleet menu truthful, which is the invariant we keep breaking.

Loading an already-loaded model silently replaces it

POST /api/models/load for a model already serving on that node doesn't stack and doesn't refuse. It reuses the instance id, tears the running engine down, and rebuilds it with the new config. The response is indistinguishable from a fresh load:

{"status":"launching","instance_id":"cefba42c:unsloth/Qwen3.8-27B-NVFP4","api_port":8000,"stacked":false}

Nothing says an in-flight instance just got stopped, and the model drops off the fleet menu for the length of the reload. It cost me about seven minutes of Qwen3.8 downtime, and it's easy to walk into: stacking is keyed on model, so "same model, different flags" is exactly what config testing looks like and exactly what triggers the destructive path.

Either return "replacing": true in the response, or require {"replace": true} and 409 without it. I don't have a strong preference between those. Silently replacing is the part that's wrong.

Docs only, no code changes.

🤖 Generated with Claude Code