Skip to content

[Benchmark] bench run --url reports every card on a split over identical GPUs - #4477

Open
coryjett wants to merge 2 commits into
Niko1221:mainfrom
coryjett:fix-bench-url-gpus
Open

coryjett wants to merge 2 commits into
Niko1221:mainfrom
coryjett:fix-bench-url-gpus

Conversation

@coryjett

@coryjett coryjett commented Oct 28, 2026 •

Copy link
Copy Markdown

Problem and result

Fixes #4473. With strata bench run --url there is no run config, so bench_facts.hardware() chooses the cards from
the server's GPU name. On a layer split the server reports every card joined by " + ", e.g.
"NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090". The code kept only cards whose name equals that whole
string. None did, so it fell back to cards[:1], and a two-card server was reported as one card.

After: one local card is taken per name in the server's " + " list, in that order. Two identical cards are both
reported, a local card the server does not use is left out, and a single served name still gives one card. A run
config's gpu selection still wins. If some served name has no local card, the old behaviour is kept.

Related discussion

#4473

Approach

A small change to the elif len(cards) > 1 and served_gpu: branch only. The served handling below it (cards only
the server can see, vendor-only names) is unchanged.

Validation

Passed:

  • python3.12 -m unittest tests.terminal.test_bench_facts_gpus tests.terminal.test_bench_facts_amd tests.terminal.test_bench_standard tests.terminal.test_bench tools.test_strata_bench:
    129 tests OK (1 skipped). Strata Control 0.1.52's bundled Python 3.12.15, Ubuntu 26.10, on this branch (main
    46c152a + this commit).
  • The new tests/terminal/test_bench_facts_gpus.py on unchanged main: 3 of 5 fail (two identical cards, a mixed
    pair, an unused third card).
  • On real hardware (2x RTX 3090, layer split, Strata 0.2.7 serving): the server's /v1/status reports
    machine.gpu.name = "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 3090". hardware() with the real
    nvidia_gpus() and that name returns two cards on this branch and one on main.
  • The original report: the --url and --profile --no-start runs in [Benchmark] 2x RTX 3090 on PCIe 3.0 x16, dual Xeon E5-2620 v3 (Dell T630), Strata 0.2.7 #4468.

Failed:

  • None observed.

Not run:

  • A full strata bench run --url with this branch (only hardware() was run on the real machine).
  • AMD/other-vendor multi-card setups beyond the unit tests (the AMD tests in test_bench_facts_amd.py pass).

Limits

  • When the server reports one name for several cards without " + " (it did not here), one card is still reported.
    hardware_static.gpu_count could cover that case; it is not used here.

🤖 Generated with Claude Code

Update: the same fix in tools/strata_bench.py

collect_machine compared the server's "A + A" name against single cards the same way, so the community runner attached to a server on two identical cards reported one card, half the VRAM total and a rtx-3090 folder slug instead of 2x-rtx-3090. Second commit: one local card per name in the server's list, mirroring bench_facts.hardware().

Passed: python3.12 -m unittest tools.test_strata_bench tests.terminal.test_bench_facts_gpus tests.terminal.test_bench_standard: Ran 96 tests in 54.014s, OK (1 skipped), same environment. The new ReportTests.test_two_identical_cards_the_server_names_are_both_listed against main's tools/strata_bench.py (production file stashed): Ran 1 test in 0.508s FAILED (failures=1)

…over identical cards (Niko1221#4473)

Without a run config (strata bench run --url) hardware() picked the local card whose name equals the server's GPU
name. A layer split reports "RTX 3090 + RTX 3090", which matches no single card, so the report fell back to the
first card. Take one local card per name in the server's " + " list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lists (Niko1221#4473)

collect_machine compared the server's "A + A" name against single cards too, so an attached server on two
identical cards got one card, half the VRAM total and a one-card folder slug.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Benchmark] bench run --url records one card on a server using two identical GPUs

1 participant