[Klaud Cold] glm5.2-fp4-b200-sglang-agentic-mtp: extend the AgentX curve to concurrency 48 with EP8/DEP8 arms / 以 EP8/DEP8 分支将 GLM-5.2 B200 SGLang AgentX 曲线扩展至并发度 48

[functionstackx]

Summary / 摘要

Extend the GLM-5.2 NVFP4 B200 SGLang AgentX MTP curve (glm5.2-fp4-b200-sglang-agentic-mtp) from concurrency 16 to 48. Past concurrency 20 the low-latency TP8 arm no longer moves the Pareto front, so the curve is now three arms, all TP8 with HiCache host-DRAM offload:

Arm	Topology	conc-list
TP8 (low-latency, existing)	tp: 8	[1, 4, 8, 12, 16, 20]
EP8 (new)	tp: 8, ep: 8	[20, 24, 28, 32, 40, 48]
DEP8 (new)	tp: 8, ep: 8, dp-attn: true + sglang-router 0.3.2	[20, 24, 28, 32, 40, 48]

将 GLM-5.2 NVFP4 B200 SGLang AgentX MTP 曲线（glm5.2-fp4-b200-sglang-agentic-mtp）的并发度从 16 扩展到 48。并发度 20 之后 TP8 低延迟分支不再改善帕累托前沿，因此曲线改为三个分支（均为 TP8 加 HiCache 主机内存卸载）：TP8 止于 20；新增 EP8（tp: 8, ep: 8）与 DEP8（tp: 8, ep: 8, dp-attn: true，经 sglang-router 0.3.2 路由）覆盖 20、24、28、32、40、48。

Changes / 改动

• configs/nvidia-master.yaml — TP8 conc-list gains 20; EP8 and DEP8 search-space entries added; entry comment rewritten.

• benchmarks/single_node/agentic/glm5.2_fp4_b200_sglang_mtp.sh

  • DEFAULT_HICACHE_SIZE=169 now also matches 20|24|28|32|40|48. The 12|16 arm is byte-identical and c1/c4/c8 still take ratio mode, so no existing point changes behavior. The 169 GB/rank pool plus the 38.73 GB/rank DSA indexer stays at about 1,662 GB across 8 ranks, inside the 1,731 GB this config exposes on cluster:b200-nscale; with attention-DP each rank pins the same pool for its own sessions.
  • The EAGLE draft-MoE override (--speculative-moe-a2a-backend none --speculative-moe-runner-backend triton) is now gated on EP_SIZE > 1 instead of DP_ATTENTION=true. Its comment already said it is needed whenever expert parallelism puts an all-to-all in the MoE path, which is true for the plain EP8 arm as well. EP_SIZE=1 (every existing point) is untouched.
  • The DP-attention branch (dp8, --enable-dp-attention, 32768 chunked prefill, sglang-router with consistent hashing on the correlation id) was already in the script and is unchanged.

• perf-changelog.yaml — appended entry marked append-only: true.

• configs/nvidia-master.yaml：TP8 conc-list 新增 20；新增 EP8 与 DEP8 搜索空间条目；重写条目注释。

• benchmarks/single_node/agentic/glm5.2_fp4_b200_sglang_mtp.sh

  • DEFAULT_HICACHE_SIZE=169 的分支新增匹配 20|24|28|32|40|48；12|16 分支逐字节不变，c1/c4/c8 仍为比例模式，现有点行为不变。每 rank 169 GB 目标池加 38.73 GB DSA 索引器在 8 rank 下约 1,662 GB，仍在本配置于 cluster:b200-nscale 暴露的 1,731 GB 之内；注意力 DP 下每个 rank 为自身会话固定同样大小的池。
  • EAGLE 草稿 MoE 覆盖（--speculative-moe-a2a-backend none --speculative-moe-runner-backend triton）改为按 EP_SIZE > 1 触发，而非 DP_ATTENTION=true。注释本已说明只要专家并行在 MoE 路径引入 all-to-all 就需要该覆盖，纯 EP8 分支同样适用。EP_SIZE=1（所有现有点）不受影响。
  • DP 注意力分支（dp8、--enable-dp-attention、32768 分块预填充、按 correlation id 一致性哈希的 sglang-router）脚本中原已存在，未改动。

• perf-changelog.yaml：追加条目并标记 append-only: true。

Verification / 验证

• generate_sweep_configs.py test-config renders 18 jobs: TP8 [1, 4, 8, 12, 16, 20], EP8 [20, 24, 28, 32, 40, 48], DEP8 [20, 24, 28, 32, 40, 48]; all steps ≥ 2.

• utils/process_changelog.py --base-ref origin/main --head-ref HEAD accepts the append-only entry and schedules exactly the 13 new points: TP8 c20, EP8 c20–c48, DEP8 c20–c48.

• bash -n on the recipe script passes.

• generate_sweep_configs.py test-config 生成 18 个任务：TP8 [1, 4, 8, 12, 16, 20]、EP8 [20, 24, 28, 32, 40, 48]、DEP8 [20, 24, 28, 32, 40, 48]，步长均 ≥ 2。

• utils/process_changelog.py --base-ref origin/main --head-ref HEAD 接受 append-only 条目，仅调度 13 个新点：TP8 c20、EP8 c20–c48、DEP8 c20–c48。

• 配方脚本 bash -n 通过。

Sibling PR #2829 extends the B300 curve to 48.

同系列 PR #2829 将 B300 曲线扩展至 48。

🤖 Generated with Claude Code

---

Note

Low Risk
Benchmark and recipe configuration only; existing conc ≤16 TP8 behavior is preserved and changes are gated on new EP/DEP topology and higher concurrencies.

Overview
Extends the GLM-5.2 NVFP4 B200 SGLang AgentX MTP benchmark (glm5.2-fp4-b200-sglang-agentic-mtp) from concurrency 16 to 48 by wiring three search-space arms in nvidia-master.yaml: low-latency TP8 through conc 20, new EP8 (tp: 8, ep: 8) and DEP8 (EP8 plus dp-attn: true behind sglang-router 0.3.2) for conc 20–48. Existing TP8 points through 16 are unchanged.

The B200 recipe script adjusts serving for those arms only: HiCache uses the 169 GB/rank absolute pool from conc 12 through 48 (not just 12/16); cookbook low-latency flags (fp8 KV, cutedsl, max-prefill) apply whenever DP attention is off, including plain EP8; and the EAGLE draft-MoE workaround (--speculative-moe-a2a-backend none, triton runner) is gated on EP_SIZE > 1 instead of DP_ATTENTION=true so EP8 initializes correctly. An append-only perf-changelog.yaml entry documents the 13 new sweep points.

^{Reviewed by Cursor Bugbot for commit dc6e956. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Code review found no issues

No high-confidence issues detected in this change.

One verified lower-impact observation (a convention, logging or cleanup point) was not posted.

This review covers commit 64e97b2, which is no longer the latest commit on this pull request; later commits are not covered by it.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33924246839
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33924246839

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33945277986
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33945277986