fix(serve): pick a free port instead of colliding on the default - #301
fix(serve): pick a free port instead of colliding on the default#301volen-silo wants to merge 1 commit into
Conversation
A second managed service started without --port was handed 11435 unconditionally, so it collided with a server already holding it. The engine only discovered the clash on bind, minutes after the model had loaded, and its dead record stayed in `rocm services list`. Resolve the address before anything is announced or launched. An omitted --port now takes the first free port at or above 11435 and reports the move; a named port is a requirement, so a taken one fails up front, before the model loads and before a service record exists. Ports claimed by live manifests count as taken alongside a bind probe, which covers a service that is still starting and has not listened yet. Reusing an existing service now reports that service's own port rather than the one that was asked for, and refuses a named port it is not on. Classify bind errors instead of reading every failure as a collision: a privileged port or an unresolvable host is quoted as itself, and does not send the automatic search scanning a range that was never going to work. The dash serve wizard no longer prefills the port, which would have made every wizard launch an explicit request and turned a second server into a hard failure. Concurrent serves can still race between the claim read and the manifest write; closing that needs the lock owned by the separate concurrency work on the managed launch path, so it is deliberately left alone. Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
cf4656d to
0869a9c
Compare
|
Closing this as superseded by #243, which covers the same defect and predates it by twelve days. I missed #243 when planning: I searched the tracker for related work and checked the pending e2e-contract PR, but never searched open PRs for the serve-port problem itself, so I built a parallel fix. My mistake — apologies for the duplicate review surface. #243 is the better change. It does everything here and more:
Two things this branch had that #243 also has, so nothing is lost: I have raised the two things I did find that #243 does not yet cover as comments over there. Nothing from this branch needs to be salvaged separately. |
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. — no row exists onmain; see Coordination below.Summary
A second managed service started without
--portwas handed 11435 unconditionally, so it collided with a server already holding it. Both manifests recorded the same address, the engine only discovered the clash when it tried to bind — minutes later, after the model had loaded — and the failed service stayed inrocm services list --all. Because the CLI advertises managing several background servers and spreads models across devices, the shared default defeated a capability the product otherwise has.Root cause: there was no port-availability check anywhere on the serve path. The default was applied by clap and passed straight through to the engine.
What changed:
--porttakes the first free port at or above 11435 and reports the move; a named port is treated as a requirement, so a taken one fails up front — before the model loads and before a service record exists, which is also what stops the dead record being left behind.Why
--portbecameOption<u16>: the two branches have to be distinguishable.--port 11435against a busy 11435 must fail, while the implicit 11435 must move; a clap default erases that distinction. Help text carries the default instead.Risk: low. No change to the launch path itself — only which port reaches it, and a new pre-flight refusal. The single user-visible behaviour change beyond the fix is that an explicitly named busy port is now an error rather than a late engine crash.
Scope
Concurrent
rocm serveinvocations can still race between the claim read and the manifest write. Closing that needs a lock held across read-select-launch, which is owned by separate in-progress concurrency work on the managed launch path; adding one here would fight it for the same critical section. The window is documented inresolve_serve_port, including the fact that the critical section starts at the claim read rather than the spawn. This is a narrowing, not a regression — the collision was previously deterministic.Test plan
--port 0rejection, unbindable-host classification, and wizard argument construction.cargo testgreen forrocm,rocm-core,rocm-dash-tui;cargo clippy --workspace --all-targetsandcargo fmtclean.serve-named-address-already-takenpins the explicit-busy-port refusal and asserts no service is recorded for it.Not verified locally, stated rather than skipped: I have no GPU on this machine, so
rocm serveexits at the GPU pre-flight before port resolution is ever reached. The original defect was reproduced on GPU hardware by the reporter, but I could not re-run that reproduction myself; the fix is verified by unit tests and by review against the reported behaviour. The new e2e scenario is@requires-gpufor the same reason scenario 13 is — on a host with no usable device the pre-flight refuses before the address is examined — so it is verified on the GPU lane, not here.Coordination
An e2e scenario for this bug is already pending in #241, along with an
expectations.tomlxfail row citing it. Neither is onmainyet, so there is no stale row for this PR to remove. Whichever merges second must reconcile: if this one lands first, #241 needs to drop that row, or the merge queue will fail on the XPASS.