Skip to content

fix(serve): pick a free port instead of colliding on the default - #301

Closed
volen-silo wants to merge 1 commit into
mainfrom
serve-picks-a-free-port-for-a-second-server
Closed

fix(serve): pick a free port instead of colliding on the default#301
volen-silo wants to merge 1 commit into
mainfrom
serve-picks-a-free-port-for-a-second-server

Conversation

@volen-silo

Copy link
Copy Markdown
Collaborator
  • If this PR fixes a bug, searched tests/e2e-cucumber/expectations.toml for the fixed ticket ID and removed/narrowed any now-stale xfail rows. — no row exists on main; see Coordination below.

Summary

A second managed service started without --port was handed 11435 unconditionally, so it collided with a server already holding it. Both manifests recorded the same address, the engine only discovered the clash when it tried to bind — minutes later, after the model had loaded — and the failed service stayed in rocm services list --all. Because the CLI advertises managing several background servers and spreads models across devices, the shared default defeated a capability the product otherwise has.

Root cause: there was no port-availability check anywhere on the serve path. The default was applied by clap and passed straight through to the engine.

What changed:

  • The address is resolved before anything is announced or launched. An omitted --port takes the first free port at or above 11435 and reports the move; a named port is treated as a requirement, so a taken one fails up front — before the model loads and before a service record exists, which is also what stops the dead record being left behind.
  • Ports claimed by live service manifests count as taken alongside a live bind probe. The manifest half covers a service that is still starting and has not listened yet — the exact state the second serve walks into.
  • Reusing an already-running service now reports that service's own port rather than the one that was asked for, and refuses a named port it is not on. Previously the plan could announce a port the reused service was not listening on.
  • Bind failures are classified rather than all read as collisions. A privileged port or an unresolvable host is quoted as itself and does not send the automatic search scanning a range that was never going to work.
  • The dash serve wizard no longer prefills the port. Prefilling made every wizard launch an explicit request, which under the new rule would turn a second server into a hard failure.

Why --port became Option<u16>: the two branches have to be distinguishable. --port 11435 against a busy 11435 must fail, while the implicit 11435 must move; a clap default erases that distinction. Help text carries the default instead.

Risk: low. No change to the launch path itself — only which port reaches it, and a new pre-flight refusal. The single user-visible behaviour change beyond the fix is that an explicitly named busy port is now an error rather than a late engine crash.

Scope

Concurrent rocm serve invocations can still race between the claim read and the manifest write. Closing that needs a lock held across read-select-launch, which is owned by separate in-progress concurrency work on the managed launch path; adding one here would fight it for the same critical section. The window is documented in resolve_serve_port, including the fact that the critical section starts at the claim read rather than the spawn. This is a narrowing, not a regression — the collision was previously deterministic.

Test plan

  • 13 new unit tests: automatic selection over a live listener and over a manifest-only claim, explicit-port refusal naming the holder, reuse reporting the live service's own port, refusal of a named port the reused service is not on, --port 0 rejection, unbindable-host classification, and wizard argument construction.
  • cargo test green for rocm, rocm-core, rocm-dash-tui; cargo clippy --workspace --all-targets and cargo fmt clean.
  • New e2e scenario serve-named-address-already-taken pins the explicit-busy-port refusal and asserts no service is recorded for it.

Not verified locally, stated rather than skipped: I have no GPU on this machine, so rocm serve exits at the GPU pre-flight before port resolution is ever reached. The original defect was reproduced on GPU hardware by the reporter, but I could not re-run that reproduction myself; the fix is verified by unit tests and by review against the reported behaviour. The new e2e scenario is @requires-gpu for the same reason scenario 13 is — on a host with no usable device the pre-flight refuses before the address is examined — so it is verified on the GPU lane, not here.

Coordination

An e2e scenario for this bug is already pending in #241, along with an expectations.toml xfail row citing it. Neither is on main yet, so there is no stale row for this PR to remove. Whichever merges second must reconcile: if this one lands first, #241 needs to drop that row, or the merge queue will fail on the XPASS.

A second managed service started without --port was handed 11435
unconditionally, so it collided with a server already holding it. The
engine only discovered the clash on bind, minutes after the model had
loaded, and its dead record stayed in `rocm services list`.

Resolve the address before anything is announced or launched. An omitted
--port now takes the first free port at or above 11435 and reports the
move; a named port is a requirement, so a taken one fails up front,
before the model loads and before a service record exists.

Ports claimed by live manifests count as taken alongside a bind probe,
which covers a service that is still starting and has not listened yet.
Reusing an existing service now reports that service's own port rather
than the one that was asked for, and refuses a named port it is not on.

Classify bind errors instead of reading every failure as a collision: a
privileged port or an unresolvable host is quoted as itself, and does not
send the automatic search scanning a range that was never going to work.

The dash serve wizard no longer prefills the port, which would have made
every wizard launch an explicit request and turned a second server into a
hard failure.

Concurrent serves can still race between the claim read and the manifest
write; closing that needs the lock owned by the separate concurrency
work on the managed launch path, so it is deliberately left alone.

Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
@volen-silo
volen-silo force-pushed the serve-picks-a-free-port-for-a-second-server branch from cf4656d to 0869a9c Compare August 24, 2026 08:45
@volen-silo

Copy link
Copy Markdown
Collaborator Author

Closing this as superseded by #243, which covers the same defect and predates it by twelve days.

I missed #243 when planning: I searched the tracker for related work and checked the pending e2e-contract PR, but never searched open PRs for the serve-port problem itself, so I built a parallel fix. My mistake — apologies for the duplicate review surface.

#243 is the better change. It does everything here and more:

  • it takes a real lease on the candidate port and holds it, rather than probing and releasing, so there is no gap between the check and the launch;
  • it serialises the whole allocation behind a cross-process lock, closing the concurrent-serve race that this PR explicitly deferred;
  • it refuses automatic selection on a host it cannot preflight, instead of probing an address the engine may not bind the same way.

Two things this branch had that #243 also has, so nothing is lost: --port 0 rejection (its clap value parser) and distinguishing a busy port from an unbindable one (it propagates the OS error rather than collapsing it).

I have raised the two things I did find that #243 does not yet cover as comments over there. Nothing from this branch needs to be salvaged separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant