Add Kimi K3 B300 1P3D EFA config - #2848
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ce0e5b7. Configure here.
| USES_EFA_NIXL=1 | ||
| EFA_NIXL_CACHE_ROOT="${B300_EFA_NIXL_CACHE_ROOT:-/data/home/sa-gha-runner/efa-nixl-1.3.2-efa-1.47.0}" | ||
| "$GITHUB_WORKSPACE/runners/setup_b300_efa_nixl.sh" \ | ||
| "$SQUASH_FILE" "$EFA_NIXL_CACHE_ROOT" "$SLURM_ACCOUNT" "$SLURM_PARTITION" |
There was a problem hiding this comment.
Overlay setup failure is ignored
High Severity
The new EFA/NIXL overlay setup is invoked without checking its exit status, and this launcher never enables set -e. A failed first-time build still leaves USES_EFA_NIXL=1 and continues into srtctl apply, so the job mounts an incomplete overlay and a 4-node B300 allocation starts without the LIBFABRIC backend it needs.
Reviewed by Cursor Bugbot for commit ce0e5b7. Configure here.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| if [[ -n "${_RECIPE_SRC:-}" ]] && grep -q '"LIBFABRIC"' "$_RECIPE_SRC"; then | ||
| USES_EFA_NIXL=1 | ||
| EFA_NIXL_CACHE_ROOT="${B300_EFA_NIXL_CACHE_ROOT:-/data/home/sa-gha-runner/efa-nixl-1.3.2-efa-1.47.0}" | ||
| "$GITHUB_WORKSPACE/runners/setup_b300_efa_nixl.sh" \ | ||
| "$SQUASH_FILE" "$EFA_NIXL_CACHE_ROOT" "$SLURM_ACCOUNT" "$SLURM_PARTITION" |
There was a problem hiding this comment.
🔴 The new call to setup_b300_efa_nixl.sh has no || exit 1, unlike every other build/import step in this file (import_squash_image, patch_srt_eval_dispatch.py, inject_synthetic_acceptance.py). Since the script has no global set -e, a failed EFA/NIXL overlay build (apt-get, EFA installer, or meson build failure) is silently ignored, USES_EFA_NIXL stays 1, and the job launches with a broken/empty overlay mounted at /nixl_overlay, failing deep inside the multi-node SLURM job instead of at build time. Fix: append || exit 1 to the setup_b300_efa_nixl.sh invocation so build failures abort the launcher immediately, matching the other invocations in this file.
Extended reasoning...
setup_b300_efa_nixl.sh runs set -euo pipefail and will exit non-zero if e.g. the EFA installer or the srun container job fails (network hiccup, apt-get transient failure, meson compile error). Because the call site at lines 213-216 lacks || exit 1 and the file has no top-level set -e (only set -x/set +x for tracing and a set -euo pipefail inside an unrelated nested subshell at line 98), the launcher proceeds past the failed call with USES_EFA_NIXL=1. It then writes default_mounts pointing at $EFA_NIXL_CACHE_ROOT/{nixl,efa,efa-system-libs}, which may be empty or partially populated, and submits the multi-node job, which fails later inside the SLURM allocation (NIXL LIBFABRIC plugin missing) instead of failing fast in the launcher with a clear error.
Verification: normal. The setup_b300_efa_nixl.sh call at runners/launch_b300-dsxe.sh:215-216 has no || exit 1 and the launcher has no top-level set -e (grep of the file shows only set -x at line 70, set +x/set -x at 316/356, and a set -euo pipefail at line 98 that is inside the srun ... bash -c "..." heredoc string of import_squash_image, not the launcher itself). So a non-zero exit from…
| @@ -0,0 +1,95 @@ | |||
| #!/usr/bin/env bash | |||
|
|
|||
| # Build the userspace pieces missing from the Kimi-K3 vLLM image for NIXL over | |||
There was a problem hiding this comment.
can efa libraries be included in the vLLM image similar to how it is already included in the pytorch image for better out of the box experience?
There was a problem hiding this comment.
Please point me to the PyTorch docker image build EFA installation, so that we can match its version across.
|
[by Codex] Closing because the AWS B300 runners are connected to NVIDIA/InferenceMAX rather than this upstream repository. Continuing the 1P3D EFA validation on the internal fork. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34076321013 |
| printf '#!/bin/sh\\nexit 0\\n' >/tmp/nixl-build-bin/git | ||
| chmod +x /tmp/nixl-build-bin/git | ||
| export PATH=/tmp/nixl-build-bin:\"\$PATH\" | ||
| meson setup /tmp/nixl-build /tmp/nixl-${NIXL_VERSION} \ |
There was a problem hiding this comment.
why are you building nixl from sources? Nixl python wheel comes with libfabric backend included.
3 participants
[by Codex]
Mirrors the 1P3D/c32 Kimi-K3 AgentX configuration from #2814 for the AWS B300 DSXE runner.
Validated with the repository matrix/changelog tools and an srt-slurm v1.0.87 dry-run.
Note
Medium Risk
Changes the B300 DSXE launch path for LIBFABRIC recipes and broadens synthetic-acceptance injection to all B300 runs, not eval-only; risk is confined to benchmark infrastructure rather than production serving.
Overview
Adds a Kimi-K3 B300 disaggregated (1 prefill / 3 decode) agentic-coding benchmark point at concurrency 32, mirroring the AgentX 1P3D setup from PR 2814 for AWS EFA: prefill-to-decode GPU KV uses NIXL over LIBFABRIC, with cuMem disabled and DMA-BUF so EFA transfers work; Mooncake host KV stays on prefill only (decode is NIXL-only).
Registers the scenario in
nvidia-master.yaml(kimik3-fp4-b300-dynamo-vllm-agentic-dspark-mooncake-dcp8-disagg-1p3d-efa-prefill-store) and documents it inperf-changelog.yaml.Extends the B300 DSXE launcher to detect recipes that reference
"LIBFABRIC", run a newsetup_b300_efa_nixl.shonce to cache NIXL 1.3.2 + EFA 1.47.0 userspace, and mount that overlay into srt-slurm jobs. Also always runsinject_synthetic_acceptance.pyon B300 launches (previously only whenEVAL_ONLY=true).Reviewed by Cursor Bugbot for commit ce0e5b7. Bugbot is set up for automated code reviews on this repo. Configure here.