Update Kimi K3 B300 AgentX / 更新 Kimi K3 B300 AgentX - #2814
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33726519593 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33726975040 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| decode_environment: | ||
| VLLM_USE_DIRECT_DCP_A2A: '1' | ||
| VLLM_USE_DIRECT_DCP_Q_GATHER: '1' | ||
| VLLM_USE_DIRECT_DCP_KV_GATHER: '1' | ||
| VLLM_ALLREDUCE_USE_FLASHINFER: '1' | ||
| VLLM_KIMI_K3_SHARD_SP_SHARED_EXPERT: '0' | ||
| DYN_REQUEST_PLANE: tcp | ||
| ETCD_LEASE_TTL: '600' | ||
| VLLM_ENGINE_READY_TIMEOUT_S: '3600' | ||
| VLLM_RPC_TIMEOUT: '600000' | ||
| TILELANG_CLEANUP_TEMP_FILES: '1' | ||
| VLLM_USE_NCCL_SYMM_MEM: '0' | ||
| NCCL_CUMEM_ENABLE: '1' | ||
| NCCL_MNNVL_ENABLE: '0' | ||
| NCCL_NVLS_ENABLE: '1' | ||
| VLLM_SERVER_DEV_MODE: '1' | ||
| VLLM_USE_V2_MODEL_RUNNER: '1' | ||
| VLLM_USE_RUST_FRONTEND: '1' | ||
| VLLM_MOONCAKE_LOAD_RECV_THREADS: '4' | ||
| MC_SLICE_SIZE: '1048576' | ||
| VLLM_CONNECTOR_PREFETCH_DEPTH: '8' | ||
| VLLM_CONNECTOR_PREFETCH_KV_CAP: '0.65' | ||
| VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS: '1800' | ||
| VLLM_PREFIX_CACHE_RETENTION_INTERVAL: '0' | ||
| NCCL_P2P_LEVEL: NVL | ||
| MC_ENABLE_DEST_DEVICE_AFFINITY: '1' | ||
| WITH_NVIDIA_PEERMEM: '0' | ||
| NCCL_NET_PLUGIN: none | ||
| UCX_MEMTYPE_CACHE: n | ||
| UCX_MEMTYPE_REG_WHOLE: n | ||
| UCX_RCACHE_MAX_UNRELEASED: '1024' | ||
| UCX_TCP_AF_PRIO: inet | ||
| VLLM_SSM_CONV_STATE_LAYOUT: DS | ||
| DG_JIT_CACHE_DIR: /tmp/dg-cache-kimi-k3-gb300-pd-dspark-mooncake-{job_id} | ||
| PYTHONHASHSEED: '42' | ||
| PYTHONNOUSERSITE: '1' | ||
| PYTHONUNBUFFERED: '1' | ||
| TORCH_CUDA_ARCH_LIST: '10.0' | ||
| MC_STORE_CLIENT_METRIC: '1' | ||
| MC_STORE_CLIENT_METRIC_INTERVAL: '5' | ||
| MC_TE_METRIC: '0' | ||
| MC_GID_INDEX: '3' | ||
| UCX_TLS: rc,cuda_copy,cuda_ipc,sm,self |
There was a problem hiding this comment.
🔴 decode_environment is missing UCX_NET_DEVICES (present in prefill_environment, line 117), so the decode worker lets UCX auto-discover NICs instead of pinning the six rail-isolated devices this whole PR exists to pin; identical omission in disagg-b300-1p2d and disagg-b300-1p3d. Fix: give decode_environment the same UCX_NET_DEVICES=mlx5_4:1,mlx5_8:1,mlx5_10:1,mlx5_16:1,mlx5_20:1,mlx5_22:1 as prefill (or share via a YAML anchor, as the existing deepseek-v4 gb200 disagg recipes do with &worker_environment/*worker_environment) in all three b300 disagg files.
Extended reasoning...
b300-nv is rail-isolated per the PR's own rationale: a QP only comes up when both peers sit on the same /24, and letting one side auto-pick a NIC caused a prior bring-up to fail with 'Active handshake RPC failed'. NIXL moves KV between prefill and decode over UCX (that's why the diff also drops the /opt/ucx-no-ud override for these jobs), so UCX on the decode side needs the same pinned rail set as prefill. Because decode_environment (lines 128-170) omits UCX_NET_DEVICES while prefill_environment (lines 84-127) sets it, decode's UCX layer falls back to auto-discovery across all mlx5 devices, which can select a device not on the pinned subnet and reproduce the same QP/RTR failure the topology check and rail pinning were added to prevent -- but only on the decode side, where nothing catches it before launch.
Verification: normal. In disagg-b300-1p1d...c48.yaml, prefill_environment pins the rails at line 117 (UCX_NET_DEVICES: mlx5_4:1,mlx5_8:1,mlx5_10:1,mlx5_16:1,mlx5_20:1,mlx5_22:1), but decode_environment (lines 128-171) lists every other UCX var (UCX_MEMTYPE_CACHE, UCX_MEMTYPE_REG_WHOLE, UCX_RCACHE_MAX_UNRELEASED, UCX_TCP_AF_PRIO at 160, UCX_TLS at 171) yet has no UCX_NET_DEVICES. The same omission is present…
| for _pair in "$SQUASH_FILE|$IMAGE" "$NGINX_SQUASH_FILE|$NGINX_IMAGE"; do | ||
| _sqsh="${_pair%%|*}"; _img="${_pair##*|}" | ||
| ( | ||
| # The three concurrency jobs start together and share this path, so | ||
| # serialize. Same shape as the dcgm-exporter import below. | ||
| exec 9> "${_sqsh}.lock" | ||
| flock -w 1800 9 | ||
| if unsquashfs -l "$_sqsh" > /dev/null 2>&1; then | ||
| echo "Squash file already exists, skipping import: $_sqsh" | ||
| exit 0 | ||
| fi | ||
| # A partial file from an earlier failed import is not a squashfs, | ||
| # and mksquashfs refuses to overwrite what it cannot parse | ||
| # ("Failed to read existing filesystem - will not overwrite"). | ||
| rm -f "$_sqsh" | ||
| enroot import -o "$_sqsh" "docker://$_img" | ||
| ) || { | ||
| echo "Error: enroot import failed for $_img" >&2 | ||
| exit 1 |
There was a problem hiding this comment.
🔴 The new Kimi-K3 squash-import subshell never checks flock's exit status, unlike the dcgm-exporter block it claims to mirror (which runs under set -euo pipefail) and the other existing squash-import flock (which does flock -w 600 9 || { ...; exit 1; }). If flock times out after 1800s under concurrent job start, the subshell proceeds unlocked into rm -f "$_sqsh"; enroot import ..., so two of the three concurrently-launched Kimi-K3 jobs can race on the same shared squash file. Fix: check flock's return value (or add set -e to the subshell) and fail fast like the sibling blocks do.
Extended reasoning...
Three Kimi-K3 sweep jobs (agg-nospec, agg-dspark, disagg) are dispatched together and all target the same $SQUASH_FILE/$NGINX_SQUASH_FILE paths. The first to grab the lock runs a slow cold enroot import of the multi-GB nightly image; if it exceeds 1800s, flock -w 1800 9 in the other jobs' subshells returns non-zero, but nothing checks that return code (no set -e, no ||), so execution falls through to unsquashfs -l (fails, file mid-write) then rm -f "$_sqsh" and a concurrent enroot import -o "$_sqsh" while the first import is still writing to it. This can corrupt the shared squash file, causing later srtctl container mounts to fail with a corrupt squashfs and burning the GPU allocation, a failure mode the base script's equivalent blocks (dcgm-exporter's set -euo pipefail srun script, and the other flock at line ~641 with explicit || exit 1) do not have.
Verification: nit. The finding is factually correct. runners/launch_b300-nv.sh sets only set -x (line 11); the new Kimi-K3 import subshell (lines 234-250) does NOT re-enable set -e, and line 237 flock -w 1800 9 has no || guard and its status is never inspected. The trailing ) || { echo "Error: enroot import failed..."; exit 1; } reacts only to the subshell's last command (the enroot import at…
| # to. Dump the newest one so the next iteration reads the real directives. | ||
| _last_slurm="$(ls -t /tmp/srtctl_*.slurm 2> /dev/null | head -1)" | ||
| if [[ -n "$_last_slurm" ]]; then | ||
| echo "--- BEGIN $_last_slurm (SBATCH directives) ---" | ||
| grep -E '^#SBATCH' "$_last_slurm" || true | ||
| echo "--- END $_last_slurm ---" |
There was a problem hiding this comment.
🟡 (optional) The new JOB_ID-failure diagnostic globs the host-wide /tmp/srtctl_*.slurm and dumps whichever file is newest by mtime, with no scoping to this job's own PID or workspace. On this shared runner host, where the PR's own sweep dispatches several concurrent B300/Kimi-K3 jobs, a failure in one job's srtctl apply can print another concurrently-submitted job's #SBATCH directives, misleading debugging of the real rejection (e.g. wrong node exclude/time-limit blamed). Fix: have srtctl (or this script) name/tag the temp script per invocation (e.g. include $$ or JOB tag) and glob only that, or capture srtctl's own stderr instead of scanning a shared /tmp.
Extended reasoning...
Base behavior before this diff: no such diagnostic existed, so no misattribution was possible. After the diff, when srtctl apply fails to yield a JOB_ID, the script runs ls -t /tmp/srtctl_*.slurm | head -1 unconditionally. If a second concurrent job on the same runner (the PR text itself describes three concurrent Kimi-K3 sweep jobs sharing this host) writes its own /tmp/srtctl_*.slurm slightly later, that file becomes 'newest' and gets dumped as this job's rejected directives, even though it belongs to an unrelated submission. An engineer investigating job A's failure now reads job B's sbatch flags and can chase the wrong root cause (e.g. conclude a node exclude or partition mismatch that never applied to job A).
Verification: nit. Line 406 _last_slurm="$(ls -t /tmp/srtctl_*.slurm 2> /dev/null | head -1)" globs the host-wide pattern and picks the newest by mtime with no scoping to this job's PID, workspace, or the temp file srtctl actually wrote for this apply. With the three Kimi-K3 sweep jobs dispatched concurrently on one runner (the same concurrency route the already-reported line-249 finding relies on),…
| - "Give Kimi-K3 a model path on b300-nv and route the agentic scenario to upstream srt-slurm with the in-repo recipes, without which multi-node Kimi-K3 exits as an unsupported model." | ||
| - "Apply synthetic acceptance on throughput runs, not only eval-only ones, so the pinned acceptance length reaches speculative-config as it already does on gb300-nv." | ||
| - "Verify each node's RDMA rail subnets before the engine starts, because a down card shifts every device index above it and the Mooncake store config is written once for the whole job." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PENDING |
There was a problem hiding this comment.
🟡 (optional) The new changelog entry's pr-link is a placeholder "https://github.com/SemiAnalysisAI/InferenceX/pull/PENDING" rather than the real PR URL, so once merged the append-only, byte-sensitive changelog permanently records a dead/incorrect link for this entry (perf-changelog.yaml is append-only, so this can't be quietly fixed later without violating that convention). Fix: replace PENDING with the actual PR number before merging, and add CI/review gating that rejects a pull/PENDING (or any non-numeric pull id) in new perf-changelog.yaml entries.
Extended reasoning...
perf-changelog.yaml documents every performance-affecting change with a pr-link field that traceability/auditing tooling or engineers rely on to jump from a changelog entry to the PR that introduced it. The new entry appended at the end of the diff sets pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PENDING, which is not a valid PR URL. Because the file is append-only and byte-sensitive per AGENTS.md, once this merges as-is, the record is permanently wrong (fixing it later means editing history in a file whose convention is to only append), leaving anyone auditing the Kimi-K3 B300 change with a broken link forever.
Verification: nit. The new changelog entry ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PENDING (perf-changelog.yaml, last line of the appended block, line 6846), whereas the preceding entry uses a valid numeric URL .../pull/2808. The placeholder literally exists in the diff and is not a valid PR URL. Per AGENTS.md the file is append-only and byte-sensitive, so once merged the…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33727323910 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33780743056 |
Add the six measured B300 points: aggregated TP8 x DCP8 no-spec at c70, aggregated DSpark-7 at c4 and c1, and 1P1D/1P2D/1P3D DCP8 x DCP8 DSpark-4 at c48/c48/c32. Recipes are translated from the bia sweep configs that measured each point; only fields b300-nv forces differ. 新增六个已实测的 B300 数据点：无投机解码的聚合 TP8 x DCP8（c70）、 DSpark-7 聚合（c4 与 c1），以及 1P1D/1P2D/1P3D 的 DCP8 x DCP8 DSpark-4 （c48/c48/c32）。recipe 由实测该点的 bia sweep config 转换而来，仅修改 b300-nv 强制要求的字段。
…b / 修复：b300-nv rail 不匹配时改为告警而非使作业失败 The pinned rail set was read off four nodes and the scheduler draws from more than four, so a mismatch means the node was never inventoried rather than that it is broken. Log it and continue. 固定的 rail 集合来自四个节点的实测，而调度器可选节点多于四个；不匹配 说明该节点未被清点，而非该节点有问题。记录日志并继续运行。
…0-nv / 修复：b300-nv 上按实际暂存位置探测 Kimi-K3 并将walltime 提升至 6 小时 The single-node Kimi-K3 arm on this cluster serves /scratch/models/Kimi-K3, so probe that first and require a non-empty directory, and fail with the list of probed paths instead of handing srtctl one that does not exist. Raise the walltime from the GB300 recipes' 04:00:00 to 06:00:00: every node pulls the ~1.5 TB checkpoint from the same shared /scratch and 1P3D has four doing it at once. batch_1 accepts at least 480 minutes. 本集群的单节点 Kimi-K3 配置使用 /scratch/models/Kimi-K3，因此优先探测该路径 并要求目录非空；未找到时直接失败并列出已探测路径。walltime 由 GB300 recipe 的 04:00:00 提升至 06:00:00：每个节点都从同一共享 /scratch 拉取约 1.5 TB 检查点，1P3D 更是四个节点同时进行。batch_1 至少允许 480 分钟。
…00 配置中显式指定 DSpark draft 采样方式 The acceptance lengths these arms pin -- 3.36 at K=4, 3.84 at K=7 -- come from the probabilistic-draft, block-verify golden curve. The other Kimi-K3 DSpark curve reads 3.22 and 3.78 at the same levels, so leaving draft_sample_method to the engine default pairs an AL with the wrong curve. Name it, as every other Kimi-K3 DSpark recipe on main does. 这些配置固定的接受长度（K=4 为 3.36，K=7 为 3.84）来自 probabilistic draft + block verify 的黄金曲线；另一条 Kimi-K3 DSpark 曲线在相同档位为 3.22 与 3.78。 若不指定 draft_sample_method，接受长度将与错误的曲线配对。
The B300 NV fleet is retired and its launcher is gone, so these arms move to cluster:b300-dsxe. That is not a relabel: the two clusters do not share a fabric. b300-nv was RoCE over Mellanox, where each rail was an mlx5 device on its own /24 and a QP only came up when both ends sat on the same one. Device indices were not stable across nodes, so the recipes named six rails explicitly and a startup script re-checked each one's subnet. b300-dsxe's compute fabric is EFA: 16 adapters per node, 400 Gb each, two per GPU, presenting no netdev and no IPv4. None of the pinned mlx5 names exist there, and there are no per-rail subnets to verify. The inventory is identical on every node checked, so there is also no unstable index to pin around. The recipes therefore drop device_name, UCX_NET_DEVICES and the RoCE GID index, and ask UCX for srd, EFA's transport, because EFA does not implement the verbs RC that rc requires. The startup script is replaced by an adapter-inventory and GPU-affinity check, since a node short of adapters runs slower than its peers rather than failing. Also drops an sbatch exclude naming a node of the retired fleet, and carries the launcher's unconditional synthetic-acceptance call into the DSXE launcher, so throughput runs get the pinned acceptance length into speculative-config as they already do on gb300-nv. B300 NV 集群已下线，其启动脚本也已移除，因此这些配置迁移到 cluster:b300-dsxe。 这并非简单改名：两个集群的网络结构完全不同。 b300-nv 使用基于 Mellanox 的 RoCE，每条 rail 是一个独立 /24 网段上的 mlx5 设备， 只有两端位于同一网段时 QP 才能建立。设备索引在各节点间并不稳定，因此配方显式指定了 六条 rail，并由启动脚本在运行前逐一校验其网段。 b300-dsxe 的计算网络是 EFA：每节点 16 个适配器，各 400 Gb，每 GPU 两个，且不提供 netdev 和 IPv4。此前固定的 mlx5 设备名在该集群并不存在，也没有可校验的 rail 网段。 所检查的各节点适配器清单完全一致，因此也不存在需要规避的索引漂移问题。配方据此移除 device_name、UCX_NET_DEVICES 和 RoCE GID index，并将 UCX_TLS 设为 srd（EFA 的传输 方式），因为 EFA 并不实现 rc 所需的 verbs RC。启动脚本改为检查适配器数量与 GPU 亲和 关系：适配器缺失的节点只会比同伴更慢，而不会直接失败。 同时移除了指向已下线集群节点的 sbatch exclude，并将启动脚本中无条件调用合成接受率 注入的改动带入 DSXE 启动脚本，使吞吐运行也能像 gb300-nv 那样把固定的接受长度写入 speculative-config。
6f987e1 to
ed2be39
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ed2be39. Configure here.
| printf ' GPU%s (bus %s):%s\n' "$idx" "$gbus" "${near:- none}" | ||
| done | ||
|
|
||
| echo "=== [b300-fabric] done ===" |
There was a problem hiding this comment.
Fabric check script never invoked
Low Severity
kimik3-b300-fabric-check.sh is never referenced. Sibling scripts in configs/ run only when a recipe sets setup_script, and none of the new B300 recipes do. The changelog says this inventory check runs before the engine starts, so the new file is dead code and the advertised EFA diagnostic never executes.
Reviewed by Cursor Bugbot for commit ed2be39. Configure here.
| search-space: | ||
| - spec-decoding: mtp | ||
| kv-offloading: dram | ||
| kv-offload-backend: { name: mooncake, version: "0.3.11.post1" } |
There was a problem hiding this comment.
Disagg Mooncake version mismatch
Medium Severity
The three disagg search-space rows pin kv-offload-backend to Mooncake 0.3.11.post1, while both B300 agg keys and the GB300 AgentX entries that use the same container lineage pin 0.3.12.post1. All six new recipes run vllm/vllm-openai:nightly-dev-x86_64-cu13-3696c77, so the disagg label cannot match the image and will mis-attribute sweep results.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit ed2be39. Configure here.
| IS_MULTINODE: 'true' | ||
| AIPERF_USE_DYNAMO_CONV_AWARE_ROUTING: '0' | ||
| AIPERF_DATASET_MMAP_CACHE_DIR: /aiperf_mmap_cache | ||
| HF_HUB_CACHE: /hf_hub_cache |
There was a problem hiding this comment.
Cache paths have no host mounts
Medium Severity
Every new recipe points AIPERF_DATASET_MMAP_CACHE_DIR at /aiperf_mmap_cache and HF_HUB_CACHE at /hf_hub_cache, and the recipe headers drop HF_HOME because of that. launch_b300-dsxe.sh writes srtslurm.yaml with no default_mounts, unlike launch_gb300-nv.sh, so those paths are not bind-mounted on this cluster.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit ed2be39. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33921996973 |
2 participants
Note
Low Risk
Changes are benchmark/Slurm recipe, launcher plumbing and sweep metadata only; they affect how B300 jobs are launched and compared, not application runtime code paths in this repo.
Overview
Adds Kimi-K3 B300 agentic-coding Slurm recipes and the matching
nvidia-masterentries forcluster:b300-nv: six points on one TP8 × DCP8 worker shape with Mooncake as the external KV tier — aggregated no-spec at c70, aggregated DSpark-7 (max-num-seqs: 2) at c4 and c1, and1p1d/1p2d/1p3dDCP8 × DCP8 DSpark-4 at c48, c48 and c32. A DCP8 worker spans two 4-GPU nodes on GB300 but fits inside one 8-GPU node here.All recipes run
vllm/vllm-openai:nightly-dev-x86_64-cu13-3696c77atgpu-memory-utilization: 0.94, Dynamoleast-loadedrouting, and thecompact_group_io/max_load_batch_keysMooncakeStoreConnectorextras. One recipe per concurrency, because the arms differ by more than concurrency: the drafting arms pinmax-num-seqs: 2and capture at 1024, the no-spec arm usesmax-num-batched-tokens: 16384withmoe-backend: auto, and only the disaggregated arms carry the NIXL connector.rejection_sample_methodstaysblockwith the acceptance length in the matrix entry (3.84 at K=7, 3.36 at K=4), soinject_synthetic_acceptance.pyrewrites it for throughput runs and the six eval jobs still verify.device_namenames six rails read off this cluster, andkimik3-b300-nic-topology.shre-checks each rail's GID subnet at startup. b300-nv is rail-isolated and a device index is not a stable rail name here — a down card shifts every index above it, so naming a shifted one puts two nodes on different subnets under one name and the QP fails half an hour into a run.MC_GID_INDEXandUCX_TLSare set because this fabric is RoCE.runners/launch_b300-nv.shgives Kimi-K3 a model path on b300-nv and routes the agentic scenario to upstream srt-slurm v1.0.36 with the in-repo recipes, the same path gb300-nv takes; without it multi-node Kimi-K3 exits as an unsupported model. The change is recorded inperf-changelog.yaml.Note
低风险
本次改动仅涉及 benchmark/Slurm recipe、launcher 管道代码与 sweep 元数据，只影响 B300 作业的启动与对比方式，不改动本仓库的应用运行时代码路径。
概述
新增 Kimi-K3 B300 agentic-coding 的 Slurm recipe 及对应的
nvidia-master条目（cluster:b300-nv）：基于同一 TP8 × DCP8 worker 形态、以 Mooncake 作为外部 KV 层的六个数据点——无投机解码的聚合配置（c70）、DSpark-7 聚合配置（max-num-seqs: 2，c4 与 c1），以及1p1d/1p2d/1p3d的 DCP8 × DCP8 DSpark-4（c48、c48、c32）。GB300 上一个 DCP8 worker 需横跨两个 4-GPU 节点，而在 B300 上可放进单个 8-GPU 节点。所有 recipe 均运行
vllm/vllm-openai:nightly-dev-x86_64-cu13-3696c77，gpu-memory-utilization: 0.94，Dynamo 路由为least-loaded，并带compact_group_io/max_load_batch_keys等MooncakeStoreConnector扩展项。每个并发一个 recipe，因为各配置的差异不止于并发数：投机解码配置固定max-num-seqs: 2且在 1024 处捕获，无投机配置使用max-num-batched-tokens: 16384与moe-backend: auto，且只有分离式配置带 NIXL connector。rejection_sample_method保持block，接受长度放在 matrix 条目中（K=7 为 3.84，K=4 为 3.36），由inject_synthetic_acceptance.py仅在吞吐运行时改写，使六个 eval 作业仍能真正校验。device_name指定本集群实测得到的六条 rail，kimik3-b300-nic-topology.sh会在启动时重新校验每条 rail 的 GID 网段。b300-nv 采用 rail 隔离，且设备索引在这里不是稳定的 rail 标识——一张卡掉线会让其上方所有索引整体位移，指定被位移的索引会让两个节点在同一名称下落到不同网段，QP 将在运行半小时后失败。MC_GID_INDEX与UCX_TLS因该 fabric 为 RoCE 而设置。runners/launch_b300-nv.sh为 b300-nv 上的 Kimi-K3 提供模型路径，并将 agentic 场景接入上游 srt-slurm v1.0.36 与仓库内的 recipe（与 gb300-nv 相同的路径）；否则多节点 Kimi-K3 会以不支持的模型退出。改动记录于perf-changelog.yaml。Note
Low Risk
Benchmark recipes, sweep metadata, and launcher injection behavior only; no application runtime code paths change.
Overview
Adds six Kimi-K3 B300 agentic-coding srt-slurm recipes and matching
nvidia-mastersweep keys forcluster:b300-dsxe: aggregated no-spec (c70), aggregated DSpark-7 at c4/c1, and disaggregated 1P1D / 1P2D / 1P3D DSpark-4 at c48/c48/c32. Recipes are ported from measured bia points but drop RoCE rail pinning (device_name,UCX_NET_DEVICES,MC_GID_INDEX) and setUCX_TLSto EFAsrd, with Mooncakeglobal_segment_size: 281GBand eval-friendly speculativerejection_sample_method: block.Registers three matrix entries that wire
CONFIG_FILEto those recipes,SYNTHETIC_ACCEPTANCElengths for throughput comparability, and documents that a TP8×DCP8 worker fits on one 8-GPU B300 node (unlike GB300’s two-node DCP8).runners/launch_b300-dsxe.shnow always runsinject_synthetic_acceptance.py(not only whenEVAL_ONLY), so throughput runs get pinned acceptance like gb300-nv. Addskimik3-b300-fabric-check.shto log EFA adapter count/state and GPU affinity before jobs start (warnings only).perf-changelog.yamlrecords the new config keys.Reviewed by Cursor Bugbot for commit ed2be39. Bugbot is set up for automated code reviews on this repo. Configure here.