Skip to content

[WIP] Add GLM-5.2-NVFP4 B300 AgentX recipes - #2654

Open
csahithi wants to merge 20 commits into
mainfrom
nv-glm5.2-fp4-b300-agentx
Open

[WIP] Add GLM-5.2-NVFP4 B300 AgentX recipes#2654
csahithi wants to merge 20 commits into
mainfrom
nv-glm5.2-fp4-b300-agentx

Conversation

@csahithi

@csahithi csahithi commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

B300 Dynamo-SGLang agentic-coding sweep for GLM-5.2-NVFP4, 7 points across two config keys:

  • glm5.2-fp4-b300-dynamo-sglang-agentic-agg: aggregated single-node TP8 (conc 2/4)
  • glm5.2-fp4-b300-dynamo-sglang-agentic-disagg: low-latency 1P2D/1P3D/1P4D/1P6D TP4-decode (conc 63/61/59/58) and throughput 2P1D DEP8-decode (conc 139)

Note

Low Risk
Changes are benchmark recipes, master config entries, changelog, and conditional CI launch logic—no production serving or security-sensitive application code.

Overview
Adds GLM-5.2-NVFP4 on B300 agentic-coding benchmarks as two new nvidia-master config keys, wired to new srt-slurm base + zip_override_mtp_agentx_lowlat recipes under benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b300-fp4/agentic/.

glm5.2-fp4-b300-dynamo-sglang-agentic-agg covers aggregated single-node TP8 at concurrency 2 and 4 (glm5.2-agentx-agg.yaml). glm5.2-fp4-b300-dynamo-sglang-agentic-disagg sweeps five disaggregated points: 1P2D–1P6D TP4 prefill/decode (conc 63/61/59/58) plus 2P1D DEP8 decode (conc 139) via glm5.2-agentx.yaml. Both use Dynamo router, EAGLE/MTP, HiCache, nixl KV transfer where disagg, and AiPerf inferencex-agentx-mvp against SemiAnalysis trace data with session affinity hooks.

runners/launch_b300-dsxe.sh gains a narrow branch for agentic glm5.2 fp4 dynamo-sglang: clone NVIDIA/srt-slurm at open PR #313 (93fae85…) instead of the usual pin, overlay only the b300-fp4 recipe subtree (not the full in-repo recipes tree), and run srtctl apply with UCX_TLS unset for glm5.2 dynamo-sglang. perf-changelog.yaml documents the seven-point sweep and image lmsysorg/sglang:nightly-dev-cu13-20260824-95f5ecd3 on b300-dsxe.

Reviewed by Cursor Bugbot for commit 6588cb2. Bugbot is set up for automated code reviews on this repo. Configure here.

B300 Dynamo-SGLang agentic-coding sweep for GLM-5.2-NVFP4, 7 points across
two config keys:

- glm5.2-fp4-b300-dynamo-sglang-agentic-agg: aggregated single-node TP8
  (conc 2/4)
- glm5.2-fp4-b300-dynamo-sglang-agentic-disagg: low-latency 1P2D/1P3D/1P4D/1P6D
  TP4-decode (conc 63/61/59/58) and throughput 2P1D DEP8-decode (conc 139)
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

Comment thread runners/launch_b300-nv.sh Outdated
Comment on lines 88 to 101
git checkout c180328b98c3793ca84a1e24a030f90545eb7d5d || exit 1
mkdir -p recipes/sglang/deepseek-v4
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4" recipes/sglang/deepseek-v4
elif [[ $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "glm5.2" && $PRECISION == "fp4" ]]; then
# GLM-5.2 B300 sglang AgentX: srt-slurm main carries the agentx-mvp scenario,
# session-affinity frontend, and custom benchmark schema these recipes need.
git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
cd "$SRT_REPO_DIR" || exit 1
git checkout main
mkdir -p recipes/sglang/glm5.2/b300-fp4
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b300-fp4" recipes/sglang/glm5.2/b300-fp4
elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "minimaxm3" && $PRECISION == "fp4" && "$CONFIG_FILE" == recipes/vllm/minimax-m3/b300-fp4/8k1k/mtp/*.yaml ]]; then
git clone --branch main --single-branch https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
cd "$SRT_REPO_DIR" || exit 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new glm5.2 elif branch in runners/launch_b300-nv.sh (lines 91-98) that clones NVIDIA/srt-slurm and stages the glm5.2 recipe files is unreachable dead code: the srt-slurm clone chain's very first branch is a bare if [[ "$IS_AGENTIC" == "1" ]]; then (line 76), and both new agentic-coding config keys set IS_AGENTIC=1, so bash always takes that catch-all instead. Fix by ANDing IS_AGENTIC==1 into the new glm5.2 condition (and moving the generic IS_AGENTIC catch-all after it), matching the ordering already used in launch_gb300-nv.sh and launch_gb200-nv.sh.

Extended reasoning...

The bug: runners/launch_b300-nv.sh builds its srt-slurm setup via an if/elif chain. The chain opens with:

if [[ "$IS_AGENTIC" == "1" ]]; then
    git clone --branch cam/sa-submission-q2-2026 --single-branch https://github.com/cquil11/srt-slurm-nv.git "$SRT_REPO_DIR"
    cd "$SRT_REPO_DIR" || exit 1
elif ... (several other elifs) ...
elif [[ $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "glm5.2" && $PRECISION == "fp4" ]]; then
    git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
    cd "$SRT_REPO_DIR" || exit 1
    git checkout main
    mkdir -p recipes/sglang/glm5.2/b300-fp4
    cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b300-fp4" recipes/sglang/glm5.2/b300-fp4

The bare IS_AGENTIC check is first and does not exclude glm5.2 in any way, so in bash's if/elif chain it always wins whenever IS_AGENTIC == 1, regardless of $FRAMEWORK/$MODEL_PREFIX/$PRECISION. The new glm5.2 elif, added later in the same chain, can therefore never execute while IS_AGENTIC=1.

Why IS_AGENTIC is always 1 for these new recipes: configs/nvidia-master.yaml adds two new config keys, glm5.2-fp4-b300-dynamo-sglang-agentic-agg and glm5.2-fp4-b300-dynamo-sglang-agentic-disagg, both declaring scenarios: agentic-coding:. .github/workflows/benchmark-multinode-tmpl.yml sets IS_AGENTIC: ${{ inputs.scenario-type == 'agentic-coding' && '1' || '0' }}. So every job spawned from these two keys runs with IS_AGENTIC=1.

Step-by-step proof of failure:

  1. A sweep point from glm5.2-fp4-b300-dynamo-sglang-agentic-agg launches with FRAMEWORK=dynamo-sglang, MODEL_PREFIX=glm5.2, PRECISION=fp4, IS_AGENTIC=1, and CONFIG_FILE=recipes/sglang/glm5.2/b300-fp4/agentic/glm5.2-agentx-agg.yaml.
  2. launch_b300-nv.sh reaches the srt-slurm clone chain. The first condition [[ "$IS_AGENTIC" == "1" ]] is true, so bash takes that branch and never evaluates the later glm5.2 elif.
  3. The cquil11/srt-slurm-nv fork (branch cam/sa-submission-q2-2026) is cloned instead of NVIDIA/srt-slurm main. No mkdir/cp -rT of any recipe happens in this branch.
  4. Later in the script: CONFIG_PATH="${CONFIG_FILE%%:*}"; then if [[ ! -f "$CONFIG_PATH" ]]; then echo "Error: CONFIG_FILE does not exist after srt-slurm setup: $CONFIG_PATH"; exit 1; fi. Since recipes/sglang/glm5.2/b300-fp4/agentic/glm5.2-agentx-agg.yaml was never staged into the cloned repo, this check fails and the job exits 1 for every one of the 7 sweep points added by this PR.

Why nothing else catches it: there's no lint or CI step validating condition ordering in this bash chain; the failure only surfaces at runtime when the missing CONFIG_PATH check fires, deep into the job after the (slow) srt-slurm clone, srtctl install, and container import have already run.

The fix, proven by sibling launchers: runners/launch_gb300-nv.sh (and launch_gb200-nv.sh) solve this exact problem correctly — every model-specific agentic branch ANDs "$IS_AGENTIC" == "1" into its own condition (e.g. line 239: elif [[ "$IS_AGENTIC" == "1" && $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "glm5.2" ]]), and the generic bare elif [[ "$IS_AGENTIC" == "1" ]] catch-all is placed LAST (line 249), after all specific cases. launch_b300-nv.sh should follow the same pattern: either AND $IS_AGENTIC == "1" into the new glm5.2 condition and move it before the generic catch-all, or move the generic if [[ "$IS_AGENTIC" == "1" ]] block to the end of the elif chain.

@github-actions

Copy link
Copy Markdown
Contributor

The srt-slurm clone chain opens with a bare `if [[ "$IS_AGENTIC" == "1" ]]`
catch-all, so the glm5.2 dynamo-sglang elif never ran for agentic jobs (all
7 GLM-5.2 B300 points hit the catch-all, cloned the wrong fork, and failed the
CONFIG_PATH check). Promote the glm5.2 branch to the opening `if` with
IS_AGENTIC==1 ANDed in and demote the generic catch-all to elif, matching the
ordering in launch_gb300-nv.sh / launch_gb200-nv.sh.
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

GLM-5.2-NVFP4 is staged under compute-node-local /scratch/models, which the
GHA login host running srtctl preflight cannot stat, so preflight fails with
"path is unavailable" even though the workers can load it. Extend the existing
dsv4 dynamo-sglang --no-preflight bypass to glm5.2 (runtime model loading still
validates the path).
The disagg decode worker's NIXL KV manager crashed at buffer registration
("VRAM memory is detected as host by UCX ... VRAM registration cannot proceed"
-> NIXL_ERR_BACKEND) because the recipe shipped no UCX config, so UCX had no
CUDA memory transport. Add UCX_TLS=cuda_copy,rc + MC_FORCE_MNNVL + NCCL_MNNVL_ENABLE
to prefill/decode env, matching the proven dsv4 B300 nixl disagg recipes.
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

Use the requested UCX_TLS override for both prefill and decode environments.
…entx

# Conflicts:
#	configs/nvidia-master.yaml
#	perf-changelog.yaml
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

将 B300 AgentX 分离式 SGLang 侧的 prefill 与 decode 环境变量中固定的 UCX_TLS 显式设置替换为 UCX_LOG_LEVEL: debug,恢复由 UCX 自行选择传输通道,同时打开 UCX 侧调试日志。
@github-actions

Copy link
Copy Markdown
Contributor

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

将 GLM-5.2 B300 AgentX 的两份 recipe 替换为上游 srt-slurm 版本(base + zip_override_mtp_agentx_lowlat),将 sglang 镜像升级到 nightly-dev-cu13-20260824-95f5ecd3,并把 nvidia-master.yaml 的 -agg 条目拆分为对应两个 zip_override 索引(c2 / c4)。
agg_nodes: 1
agg_workers: 1
gpus_per_decode: 8
gpus_per_prefill: 8

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aggregated recipe missing GPU field

High Severity

The aggregated zip override sets agg_nodes/agg_workers but uses gpus_per_decode and gpus_per_prefill instead of gpus_per_agg. Every other aggregated recipe in-repo (GB200/GB300 GLM, DeepSeek H200/GB300) uses gpus_per_agg. Aggregated worker GPU allocation is likely wrong or rejected.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e8c0a18. Configure here.

enable-dp-attention: [false, false, false, false, true]
enable-dp-lm-head: [false, false, false, false, true]
max-running-requests: [16, 16, 16, 16, 139]
tensor-parallel-size: [4, 4, 4, 4, 8]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DEP8 decode missing expert parallel

High Severity

The throughput point enables decode data-parallel-size: 8 with enable-dp-attention: true but never overrides expert-parallel-size, which stays at base 1. Master config also labels that decode arm as ep: 1 while calling it DEP8. Sibling GLM DEP recipes keep ep equal to dp under dp-attention.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e8c0a18. Configure here.

合并 origin/main,将 perf-changelog.yaml 解析为 main 的完整条目列表并把本分支的 PR #2654 条目附加在末尾。

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

hshrivastava-droid and others added 4 commits September 10, 2026 14:33
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…entx-runner-update

# Conflicts:
#	configs/nvidia-master.yaml
#	perf-changelog.yaml
#	runners/launch_b300-nv.sh
launch_b300-nv.sh was retired on main (#2826) in favor of the B300 DSXE
cluster, so PR #2654's runner move to cluster:b300-dsxe needs the GLM-5.2
AgentX special-casing carried over: the pinned NVIDIA/srt-slurm#313 fork
checkout (scoped to just the glm5.2/b300-fp4 recipes), and the UCX_TLS-unset
wrapper around srtctl apply for dynamo-sglang.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Master config now rejects disagg=false entries that use prefill/decode
instead of a single worker: block (validated by
infx.matrix.validation.MultiNodeMasterConfigEntry, which landed on main
after this PR's branch point). Convert
glm5.2-fp4-b300-dynamo-sglang-agentic-agg's two search-space points to
worker:/num-nodes: to match, mirroring the sibling gb200-mtp-agg entry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6588cb2. Configure here.

--ui simple \
--server-metrics-formats json \
--output-artifact-dir /logs/aiperf
rm -f /logs/aiperf/inputs.json

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recipes skip InferenceX agentic harness

High Severity

Both recipes invoke /aiperf-venv/bin/aiperf inline and write raw artifacts to /logs/aiperf instead of running agentic_srt.sh. The multinode workflow requires ${RESULT_FILENAME}_conc*.json files with num_requests_successful, which only the InferenceX harness writes. Every point in this sweep will fail result collection even if aiperf itself succeeds.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6588cb2. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants