[WIP] Add GLM-5.2-NVFP4 B300 AgentX recipes - #2654
Conversation
B300 Dynamo-SGLang agentic-coding sweep for GLM-5.2-NVFP4, 7 points across two config keys: - glm5.2-fp4-b300-dynamo-sglang-agentic-agg: aggregated single-node TP8 (conc 2/4) - glm5.2-fp4-b300-dynamo-sglang-agentic-disagg: low-latency 1P2D/1P3D/1P4D/1P6D TP4-decode (conc 63/61/59/58) and throughput 2P1D DEP8-decode (conc 139)
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
| git checkout c180328b98c3793ca84a1e24a030f90545eb7d5d || exit 1 | ||
| mkdir -p recipes/sglang/deepseek-v4 | ||
| cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4" recipes/sglang/deepseek-v4 | ||
| elif [[ $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "glm5.2" && $PRECISION == "fp4" ]]; then | ||
| # GLM-5.2 B300 sglang AgentX: srt-slurm main carries the agentx-mvp scenario, | ||
| # session-affinity frontend, and custom benchmark schema these recipes need. | ||
| git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR" | ||
| cd "$SRT_REPO_DIR" || exit 1 | ||
| git checkout main | ||
| mkdir -p recipes/sglang/glm5.2/b300-fp4 | ||
| cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b300-fp4" recipes/sglang/glm5.2/b300-fp4 | ||
| elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "minimaxm3" && $PRECISION == "fp4" && "$CONFIG_FILE" == recipes/vllm/minimax-m3/b300-fp4/8k1k/mtp/*.yaml ]]; then | ||
| git clone --branch main --single-branch https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR" | ||
| cd "$SRT_REPO_DIR" || exit 1 |
There was a problem hiding this comment.
🔴 The new glm5.2 elif branch in runners/launch_b300-nv.sh (lines 91-98) that clones NVIDIA/srt-slurm and stages the glm5.2 recipe files is unreachable dead code: the srt-slurm clone chain's very first branch is a bare if [[ "$IS_AGENTIC" == "1" ]]; then (line 76), and both new agentic-coding config keys set IS_AGENTIC=1, so bash always takes that catch-all instead. Fix by ANDing IS_AGENTIC==1 into the new glm5.2 condition (and moving the generic IS_AGENTIC catch-all after it), matching the ordering already used in launch_gb300-nv.sh and launch_gb200-nv.sh.
Extended reasoning...
The bug: runners/launch_b300-nv.sh builds its srt-slurm setup via an if/elif chain. The chain opens with:
if [[ "$IS_AGENTIC" == "1" ]]; then
git clone --branch cam/sa-submission-q2-2026 --single-branch https://github.com/cquil11/srt-slurm-nv.git "$SRT_REPO_DIR"
cd "$SRT_REPO_DIR" || exit 1
elif ... (several other elifs) ...
elif [[ $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "glm5.2" && $PRECISION == "fp4" ]]; then
git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
cd "$SRT_REPO_DIR" || exit 1
git checkout main
mkdir -p recipes/sglang/glm5.2/b300-fp4
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b300-fp4" recipes/sglang/glm5.2/b300-fp4The bare IS_AGENTIC check is first and does not exclude glm5.2 in any way, so in bash's if/elif chain it always wins whenever IS_AGENTIC == 1, regardless of $FRAMEWORK/$MODEL_PREFIX/$PRECISION. The new glm5.2 elif, added later in the same chain, can therefore never execute while IS_AGENTIC=1.
Why IS_AGENTIC is always 1 for these new recipes: configs/nvidia-master.yaml adds two new config keys, glm5.2-fp4-b300-dynamo-sglang-agentic-agg and glm5.2-fp4-b300-dynamo-sglang-agentic-disagg, both declaring scenarios: agentic-coding:. .github/workflows/benchmark-multinode-tmpl.yml sets IS_AGENTIC: ${{ inputs.scenario-type == 'agentic-coding' && '1' || '0' }}. So every job spawned from these two keys runs with IS_AGENTIC=1.
Step-by-step proof of failure:
- A sweep point from
glm5.2-fp4-b300-dynamo-sglang-agentic-agglaunches with FRAMEWORK=dynamo-sglang, MODEL_PREFIX=glm5.2, PRECISION=fp4, IS_AGENTIC=1, and CONFIG_FILE=recipes/sglang/glm5.2/b300-fp4/agentic/glm5.2-agentx-agg.yaml. launch_b300-nv.shreaches the srt-slurm clone chain. The first condition[[ "$IS_AGENTIC" == "1" ]]is true, so bash takes that branch and never evaluates the later glm5.2 elif.- The cquil11/srt-slurm-nv fork (branch cam/sa-submission-q2-2026) is cloned instead of NVIDIA/srt-slurm main. No mkdir/cp -rT of any recipe happens in this branch.
- Later in the script:
CONFIG_PATH="${CONFIG_FILE%%:*}"; thenif [[ ! -f "$CONFIG_PATH" ]]; then echo "Error: CONFIG_FILE does not exist after srt-slurm setup: $CONFIG_PATH"; exit 1; fi. Sincerecipes/sglang/glm5.2/b300-fp4/agentic/glm5.2-agentx-agg.yamlwas never staged into the cloned repo, this check fails and the job exits 1 for every one of the 7 sweep points added by this PR.
Why nothing else catches it: there's no lint or CI step validating condition ordering in this bash chain; the failure only surfaces at runtime when the missing CONFIG_PATH check fires, deep into the job after the (slow) srt-slurm clone, srtctl install, and container import have already run.
The fix, proven by sibling launchers: runners/launch_gb300-nv.sh (and launch_gb200-nv.sh) solve this exact problem correctly — every model-specific agentic branch ANDs "$IS_AGENTIC" == "1" into its own condition (e.g. line 239: elif [[ "$IS_AGENTIC" == "1" && $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "glm5.2" ]]), and the generic bare elif [[ "$IS_AGENTIC" == "1" ]] catch-all is placed LAST (line 249), after all specific cases. launch_b300-nv.sh should follow the same pattern: either AND $IS_AGENTIC == "1" into the new glm5.2 condition and move it before the generic catch-all, or move the generic if [[ "$IS_AGENTIC" == "1" ]] block to the end of the elif chain.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32165814158 |
The srt-slurm clone chain opens with a bare `if [[ "$IS_AGENTIC" == "1" ]]` catch-all, so the glm5.2 dynamo-sglang elif never ran for agentic jobs (all 7 GLM-5.2 B300 points hit the catch-all, cloned the wrong fork, and failed the CONFIG_PATH check). Promote the glm5.2 branch to the opening `if` with IS_AGENTIC==1 ANDed in and demote the generic catch-all to elif, matching the ordering in launch_gb300-nv.sh / launch_gb200-nv.sh.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32167101624 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32167548413 |
GLM-5.2-NVFP4 is staged under compute-node-local /scratch/models, which the GHA login host running srtctl preflight cannot stat, so preflight fails with "path is unavailable" even though the workers can load it. Extend the existing dsv4 dynamo-sglang --no-preflight bypass to glm5.2 (runtime model loading still validates the path).
…entx # Conflicts: # perf-changelog.yaml
The disagg decode worker's NIXL KV manager crashed at buffer registration
("VRAM memory is detected as host by UCX ... VRAM registration cannot proceed"
-> NIXL_ERR_BACKEND) because the recipe shipped no UCX config, so UCX had no
CUDA memory transport. Add UCX_TLS=cuda_copy,rc + MC_FORCE_MNNVL + NCCL_MNNVL_ENABLE
to prefill/decode env, matching the proven dsv4 B300 nixl disagg recipes.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32174659553 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32179040742 |
Use the requested UCX_TLS override for both prefill and decode environments.
…entx # Conflicts: # perf-changelog.yaml
…entx # Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32281705398 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32307702676 |
将 B300 AgentX 分离式 SGLang 侧的 prefill 与 decode 环境变量中固定的 UCX_TLS 显式设置替换为 UCX_LOG_LEVEL: debug,恢复由 UCX 自行选择传输通道,同时打开 UCX 侧调试日志。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32410588562 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32410588562 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32435060137 |
将 GLM-5.2 B300 AgentX 的两份 recipe 替换为上游 srt-slurm 版本(base + zip_override_mtp_agentx_lowlat),将 sglang 镜像升级到 nightly-dev-cu13-20260824-95f5ecd3,并把 nvidia-master.yaml 的 -agg 条目拆分为对应两个 zip_override 索引(c2 / c4)。
| agg_nodes: 1 | ||
| agg_workers: 1 | ||
| gpus_per_decode: 8 | ||
| gpus_per_prefill: 8 |
There was a problem hiding this comment.
Aggregated recipe missing GPU field
High Severity
The aggregated zip override sets agg_nodes/agg_workers but uses gpus_per_decode and gpus_per_prefill instead of gpus_per_agg. Every other aggregated recipe in-repo (GB200/GB300 GLM, DeepSeek H200/GB300) uses gpus_per_agg. Aggregated worker GPU allocation is likely wrong or rejected.
Reviewed by Cursor Bugbot for commit e8c0a18. Configure here.
| enable-dp-attention: [false, false, false, false, true] | ||
| enable-dp-lm-head: [false, false, false, false, true] | ||
| max-running-requests: [16, 16, 16, 16, 139] | ||
| tensor-parallel-size: [4, 4, 4, 4, 8] |
There was a problem hiding this comment.
DEP8 decode missing expert parallel
High Severity
The throughput point enables decode data-parallel-size: 8 with enable-dp-attention: true but never overrides expert-parallel-size, which stays at base 1. Master config also labels that decode arm as ep: 1 while calling it DEP8. Sibling GLM DEP recipes keep ep equal to dp under dp-attention.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit e8c0a18. Configure here.
合并 origin/main,将 perf-changelog.yaml 解析为 main 的完整条目列表并把本分支的 PR #2654 条目附加在末尾。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32836132886 |
修复 GLM-5.2 B300 AgentX 配方镜像解析。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32912628075 |
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…entx-runner-update # Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml # runners/launch_b300-nv.sh
launch_b300-nv.sh was retired on main (#2826) in favor of the B300 DSXE cluster, so PR #2654's runner move to cluster:b300-dsxe needs the GLM-5.2 AgentX special-casing carried over: the pinned NVIDIA/srt-slurm#313 fork checkout (scoped to just the glm5.2/b300-fp4 recipes), and the UCX_TLS-unset wrapper around srtctl apply for dynamo-sglang. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Master config now rejects disagg=false entries that use prefill/decode instead of a single worker: block (validated by infx.matrix.validation.MultiNodeMasterConfigEntry, which landed on main after this PR's branch point). Convert glm5.2-fp4-b300-dynamo-sglang-agentic-agg's two search-space points to worker:/num-nodes: to match, mirroring the sibling gb200-mtp-agg entry. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6588cb2. Configure here.
| --ui simple \ | ||
| --server-metrics-formats json \ | ||
| --output-artifact-dir /logs/aiperf | ||
| rm -f /logs/aiperf/inputs.json |
There was a problem hiding this comment.
Recipes skip InferenceX agentic harness
High Severity
Both recipes invoke /aiperf-venv/bin/aiperf inline and write raw artifacts to /logs/aiperf instead of running agentic_srt.sh. The multinode workflow requires ${RESULT_FILENAME}_conc*.json files with num_requests_successful, which only the InferenceX harness writes. Every point in this sweep will fail result collection even if aiperf itself succeeds.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 6588cb2. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34533625594 |


B300 Dynamo-SGLang agentic-coding sweep for GLM-5.2-NVFP4, 7 points across two config keys:
Note
Low Risk
Changes are benchmark recipes, master config entries, changelog, and conditional CI launch logic—no production serving or security-sensitive application code.
Overview
Adds GLM-5.2-NVFP4 on B300 agentic-coding benchmarks as two new
nvidia-masterconfig keys, wired to new srt-slurm base +zip_override_mtp_agentx_lowlatrecipes underbenchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b300-fp4/agentic/.glm5.2-fp4-b300-dynamo-sglang-agentic-aggcovers aggregated single-node TP8 at concurrency 2 and 4 (glm5.2-agentx-agg.yaml).glm5.2-fp4-b300-dynamo-sglang-agentic-disaggsweeps five disaggregated points: 1P2D–1P6D TP4 prefill/decode (conc 63/61/59/58) plus 2P1D DEP8 decode (conc 139) viaglm5.2-agentx.yaml. Both use Dynamo router, EAGLE/MTP, HiCache, nixl KV transfer where disagg, and AiPerfinferencex-agentx-mvpagainst SemiAnalysis trace data with session affinity hooks.runners/launch_b300-dsxe.shgains a narrow branch for agentic glm5.2 fp4 dynamo-sglang: clone NVIDIA/srt-slurm at open PR #313 (93fae85…) instead of the usual pin, overlay only the b300-fp4 recipe subtree (not the full in-repo recipes tree), and runsrtctl applywithUCX_TLSunset for glm5.2 dynamo-sglang.perf-changelog.yamldocuments the seven-point sweep and imagelmsysorg/sglang:nightly-dev-cu13-20260824-95f5ecd3onb300-dsxe.Reviewed by Cursor Bugbot for commit 6588cb2. Bugbot is set up for automated code reviews on this repo. Configure here.