[Klaud Cold] Update qwen3.5-fp8-h200-sglang-agentic-hicache-mtp SGLang image to v0.5.19-cu130 / 将 qwen3.5-fp8-h200-sglang-agentic-hicache-mtp 的 SGLang 镜像更新至 v0.5.19-cu130 - #2966
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Initial attempt
Throughput is flat; per-token latency at this lowest-concurrency point is a few percent slower on the release. Smoke is compatibility evidence only, not a full curve. gsm8k: N/A in test-config mode (no agentic eval selected); it runs in the final sweep.
初始尝试
吞吐持平；该最低并发点上发布版的逐 token 延迟慢几个百分点。Smoke 仅为兼容性证据，不是完整曲线。gsm8k：test-config 模式下为 N/A（未选择 agentic 评测）；将在最终 sweep 中运行。
|
…pdate to v0.5.19-cu130 Append the perf-changelog entry for moving the H200 Qwen3.5 FP8 SGLang AgentX HiCache MTP recipe to lmsysorg/sglang:v0.5.19-cu130 (PR #2966). 为将 H200 Qwen3.5 FP8 SGLang AgentX HiCache MTP 配方切换到 lmsysorg/sglang:v0.5.19-cu130（PR #2966）追加 perf-changelog 条目。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Final full sweep
Throughput is flat to slightly lower (worst -3.0% output tok/s/GPU at concurrency 8, +1.8% at concurrency 4). Latency regresses mildly on the release: median TPOT is 0.4-5.1% slower at every point, median TTFT is slower at 6 of 8 points (up to +6.9% at concurrency 12), median E2E is 0.7-7.6% slower everywhere. These are reported as regressions without a rejection threshold; the release trades the nightly-only Qwen3.5 GDN prefill and EAGLE optimizations listed in the Initial attempt for a tagged, reproducible engine. Server-reported GPU cache hit rate is not compared: this sweep's values exceed 1.0 at concurrency 12-24, so the counter is not directly comparable to the baseline's 0.94-0.99.
最终完整 sweep
吞吐持平或略降（最差为并发 8 的输出 tok/s/GPU -3.0%，并发 4 为 +1.8%）。发布版的延迟轻微回退：TPOT 中位数在每个点慢 0.4-5.1%，TTFT 中位数在 8 个点中的 6 个变慢（并发 12 最多 +6.9%），端到端中位数各点慢 0.7-7.6%。以上作为回退如实报告，不设拒绝阈值；发布版以初始尝试中列出的 nightly 独有 Qwen3.5 GDN prefill 与 EAGLE 优化，换取带标签、可复现的引擎。服务器上报的 GPU 缓存命中率不作比较：本次 sweep 在并发 12-24 的值超过 1.0，该计数器与基线的 0.94-0.99 不可直接比较。
|
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34486082107 |
Repair 1/5
| Conc | Output tok/s/GPU | Input tok/s/GPU | Median TTFT (s) | Median TPOT (ms) | Median E2E (s) | profiled/errors | power | Throughput is flat to slightly lower (worst -4.4% output tok/s/GPU at concurrency 8; -2.3% at 4). Latency regresses mildly on the release: median TPOT is 0.4-5.5% slower at every point, median TTFT slower at 6 of 8 points (up to +9.3% at concurrency 2), median E2E slower at 7 of 8 (up to +6.5% at concurrency 12). The concurrency-24 point dropped 2 of 3600 profiled records as errors (ClientOSError: 2); the earlier green sweep on
修复 1/5
| Conc | Output tok/s/GPU | Input tok/s/GPU | Median TTFT (s) | Median TPOT (ms) | Median E2E (s) | profiled/errors | power | 吞吐持平或略降（最差为并发 8 的输出 tok/s/GPU -4.4%；并发 4 为 -2.3%）。发布版延迟轻微回退：TPOT 中位数在每个点慢 0.4-5.5%，TTFT 中位数在 8 个点中的 6 个变慢（并发 2 最多 +9.3%），端到端中位数在 8 个点中的 7 个变慢（并发 12 最多 +6.5%）。并发 24 点在 3600 条计入记录中有 2 条因错误被丢弃（ClientOSError: 2）；此前
|
…ge to v0.5.19-cu130 Move the H200 Qwen3.5 FP8 SGLang AgentX HiCache MTP recipe from the lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 dev nightly to the v0.5.19 release image lmsysorg/sglang:v0.5.19-cu130 (digest sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9, build commit sgl-project/sglang@0bcd822). Model, TP8/EP1 topology, EAGLE MTP settings, DRAM HiCache offload, concurrency list and the launch script are unchanged. 将 H200 Qwen3.5 FP8 SGLang AgentX HiCache MTP 配方的镜像从开发 nightly lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 切换到 v0.5.19 发布版 lmsysorg/sglang:v0.5.19-cu130（摘要 sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9， 构建提交 sgl-project/sglang@0bcd822）。 模型、TP8/EP1 拓扑、EAGLE MTP 设置、DRAM HiCache 卸载、并发列表和启动脚本均保持不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pdate to v0.5.19-cu130 Append the perf-changelog entry for moving the H200 Qwen3.5 FP8 SGLang AgentX HiCache MTP recipe to lmsysorg/sglang:v0.5.19-cu130 (PR #2966), rebased onto current main after other changelog entries landed. The entry selects the whole family without scenario, eval-selection or append-only modifiers. 为将 H200 Qwen3.5 FP8 SGLang AgentX HiCache MTP 配方切换到 lmsysorg/sglang:v0.5.19-cu130（PR #2966）追加 perf-changelog 条目；在其他 changelog 条目合入后已重新基于当前 main。该条目选择整个配方族，不带 scenario、评测选择或 append-only 修饰符。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
4531276 to
8604d93
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34504764956 |
|
Klaud Cold: validated. All owned runs are terminal. Repairs: 1. Runs: 34477898041, 34486032361, 34486082107, 34504699566, 34504764956. The full sweep is verified; this PR remains ready for review. Klaud Cold：validated。所有自有运行均已结束。修复次数：1。运行：34477898041, 34486032361, 34486082107, 34504699566, 34504764956。 完整 sweep 已通过验证；PR 保持就绪，等待审查。 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because configs/nvidia-master.yaml has named CODEOWNERs (ankur-singh, kedarpotdar-nv) beyond the default wildcard, a human look would still be worthwhile before merging.
What was reviewed: the single-line image bump in configs/nvidia-master.yaml for the qwen3.5-fp8-h200-sglang-agentic-hicache-mtp recipe, and the corresponding append-only perf-changelog.yaml entry (confirmed it only adds 7 lines at EOF, no historical bytes touched). Checked whether the new image: value carries a @ sha256:... digest matching the one cited in the changelog text (it does not — the config uses the bare tag lmsysorg/sglang:v0.5.19-cu130), and whether losing the old nightly tag's baked-in date/commit specificity matters for reproducibility; both were already flagged as investigated/ruled-out candidates rather than new findings.
Extended reasoning...
This is a two-file, mechanical-looking change: a single image: tag bump in configs/nvidia-master.yaml plus a corresponding append to perf-changelog.yaml. The diff is minimal and follows the repository's documented image-bump and append-only changelog conventions (verified the changelog addition is strictly appended at EOF with no modification to prior entries, and that no model.container field exists in this single-node recipe to keep in sync).
No security risks are present — this is a config data change with no code execution paths, credentials, or auth logic involved.
The bug-hunting system reported zero findings, and two candidate issues (digest mismatch between the changelog's claimed sha256 pin and the actual undigested image tag; loss of the old nightly tag's baked-in immutability) were investigated and explicitly ruled out rather than raised as findings. Despite the low complexity, the CODEOWNERS file designates specific named owners (ankur-singh, kedarpotdar-nv) for configs/nvidia-master.yaml beyond the default wildcard team, which per the review guidelines means a human should still weigh in rather than approving outright.
No outstanding CHANGES_REQUESTED or unaddressed third-party objections were visible in the available timeline metadata, and I am not restating any inline findings since none exist. Given the CODEOWNERS designation, deferring rather than approving is the appropriate call here.
|
/reuse-sweep-run 34504764956 |
合并 main 到 PR #2966，保留已验证的配置并复用完整 sweep 结果。
2 participants
Move the
qwen3.5-fp8-h200-sglang-agentic-hicache-mtpmaster image from the dev nightlylmsysorg/sglang:nightly-dev-cu13-20260907-30705c00to the current SGLang releaselmsysorg/sglang:v0.5.19-cu130(Docker Hub digestsha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9, image labelai.sglang.build.commit= sgl-project/sglang@0bcd822, tag v0.5.19). Model, TP8/EP1 topology, EAGLE MTP settings, DRAM HiCache offload, the concurrency list andbenchmarks/single_node/agentic/qwen3.5_fp8_h200_mtp.share unchanged.Baseline
GET /api/v1/benchmarks?model=Qwen-3.5-397B-A17B&date=2026-09-08&exact=true,GET /api/v1/workflow-info?date=2026-09-08&benchmarkType=agentic_traces,GET /api/v1/evaluations?model=Qwen-3.5-397B-A17B)lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00(digestsha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007, image label build commit sgl-project/sglang@30705c00, main as of 2026-09-07)cluster:h200-dgxc),Qwen/Qwen3.5-397B-A17B-FP8, SGLang FP8 with fp8_e4m3 KV, TP8 EP1, EAGLE MTP (3 steps, top-k 1, 4 draft tokens, golden acceptance length 3.39), DRAM HiCache offload, Agentic Traces (agentic-coding, with-subagents 256k corpus), concurrency 2 / 4 / 8 / 10 / 12 / 16 / 20 / 24, recipe fingerprint8b0a0360a358c71d6f9c553603f93792cab395d86dbf8ac73b5b1be735e6378d9eb6d08a30711f971c6a5e8577183720c96851dc, changelog PR #2868); benchmark result IDs 441163, 441151, 441164, 441160, 441162, 441154, 441159, 441155. The logicalcurve_workflow_run_id2408 is a snapshot ID, not the producer.em_strict0.9773 /em_flexible0.9765 (evaluation ID 10031, same producer run). The public evaluations feed labels this rowdisagg: true, which does not match the single-node TP8 recipe; the benchmark rows above carry the correctdisagg: falseidentity.将
qwen3.5-fp8-h200-sglang-agentic-hicache-mtp的主配置镜像从开发 nightlylmsysorg/sglang:nightly-dev-cu13-20260907-30705c00切换到当前 SGLang 发布版lmsysorg/sglang:v0.5.19-cu130（Docker Hub 摘要sha256:d6e7288627be8b02be88e4bba38e73f6d50e2826869f753c13a4c4385ab3eda9，镜像标签ai.sglang.build.commit= sgl-project/sglang@0bcd822，标签 v0.5.19）。模型、TP8/EP1 拓扑、EAGLE MTP 设置、DRAM HiCache 卸载、并发列表以及benchmarks/single_node/agentic/qwen3.5_fp8_h200_mtp.sh均保持不变。基线
GET /api/v1/benchmarks?model=Qwen-3.5-397B-A17B&date=2026-09-08&exact=true、GET /api/v1/workflow-info?date=2026-09-08&benchmarkType=agentic_traces、GET /api/v1/evaluations?model=Qwen-3.5-397B-A17B）lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00（摘要sha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007，镜像标签构建提交 sgl-project/sglang@30705c00，即 2026-09-07 的 main）cluster:h200-dgxc），Qwen/Qwen3.5-397B-A17B-FP8，SGLang FP8 与 fp8_e4m3 KV，TP8 EP1，EAGLE MTP（3 步、top-k 1、4 个草稿 token、黄金接受长度 3.39），DRAM HiCache 卸载，Agentic Traces（agentic-coding，含子代理的 256k 语料），并发 2 / 4 / 8 / 10 / 12 / 16 / 20 / 24，配方指纹8b0a0360a358c71d6f9c553603f93792cab395d86dbf8ac73b5b1be735e6378d9eb6d08a30711f971c6a5e8577183720c96851dc，changelog PR #2868）；基准结果 ID 441163、441151、441164、441160、441162、441154、441159、441155。逻辑curve_workflow_run_id2408 是快照 ID，不是生产运行。em_strict0.9773 /em_flexible0.9765（评测 ID 10031，同一数据来源运行）。公共评测接口将该行标记为disagg: true，与单节点 TP8 配方不符；上表基准行的disagg: false身份是正确的。🤖 Generated with Claude Code
Note
Low Risk
Benchmark config and changelog only; no application or infra logic changes beyond the SGLang container image pin.
Overview
Pins the
qwen3.5-fp8-h200-sglang-agentic-hicache-mtpmaster config from the cu13 dev nightlylmsysorg/sglang:nightly-dev-cu13-20260907-30705c00to the release imagelmsysorg/sglang:v0.5.19-cu130.Adds a
perf-changelog.yamlentry for that config key documenting the image swap (digests/build commits) and stating that model, TP8/EP1 DRAM HiCache topology, EAGLE MTP settings, concurrency sweep, andbenchmarks/single_node/agentic/qwen3.5_fp8_h200_mtp.share unchanged.Reviewed by Cursor Bugbot for commit cf5902f. Bugbot is set up for automated code reviews on this repo. Configure here.