flydsl qr int4

[samremes]

No description provided.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 2 --add-label <label>

PR title tags:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf] and op tags like [MLA] are left untouched. Add the no-auto-title label to opt this PR out of title tagging.

[Copilot]

Pull request overview

Adds a gfx942 TP8 FlyDSL INT4 QuickReduce all-reduce implementation.

Changes:

• Implements the INT4/E4M3 two-shot GPU kernel and host launcher.
• Adds uncached HIP IPC buffer management.
• Exports QRInt4 and adds correctness/benchmark coverage.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
op_tests/flydsl_tests/test_flydsl_qr_int4.py	Adds TP8 correctness and benchmark tests.
aiter/ops/flydsl/kernels/qr_int4.py	Adds the public host-side engine.
aiter/ops/flydsl/kernels/qr_int4_kernel.py	Implements the FlyDSL reduction kernel.
aiter/ops/flydsl/kernels/qr_int4_ipc.py	Implements HIP IPC allocation and exchange.
aiter/ops/flydsl/__init__.py	Exports QRInt4.

Suppressed comments (4)

aiter/ops/flydsl/kernels/qr_int4.py:177

• allreduce forwards raw data_ptr() values but only validates dtype and byte count. A CPU bf16 tensor or a non-dense view such as base[:, ::2] passes these checks, so the GPU kernel receives an invalid pointer or reduces the wrong physical bytes. Require CUDA tensors on the engine's device and dense (or explicitly supported weak-contiguous) input/output layouts before launch.

    def allreduce(self, inp: torch.Tensor, out: torch.Tensor, stream=None):
        if inp.dtype != torch.bfloat16 or out.dtype != torch.bfloat16:
            raise ValueError("QRInt4 supports bf16 input/output")
        live_bytes = int(inp.numel()) * int(inp.element_size())
        if live_bytes % 16 != 0:
            raise ValueError("byte size must be a multiple of 16 (8 bf16)")
        if int(out.numel()) * int(out.element_size()) != live_bytes:
            raise ValueError("inp/out byte size mismatch")

aiter/ops/flydsl/kernels/qr_int4.py:139

• Every launch reuses the engine's color counters and the same IPC inbox. Two host calls submitted to different CUDA streams are unordered, so both kernels can observe the same color and overwrite the same phase slots, causing incorrect output or a collective hang. Serialize launches with an event/stream wait, or explicitly reject changing streams for an engine.

        if stream is None:
            stream = torch.cuda.current_stream()

op_tests/flydsl_tests/test_flydsl_qr_int4.py:258

• The payload contains results from all eight ranks, but this return discards ranks 1–7, so every SQNR and super-tile assertion only checks rank 0. A rank-dependent peer-map or gather defect can therefore pass the new correctness test. Return or aggregate every rank's rows and assert each rank.

    return payload["ranks"][0]

op_tests/flydsl_tests/test_flydsl_qr_int4.py:207

• Removing HIP_VISIBLE_DEVICES discards the GPU set assigned by a scheduler. For example, a job assigned physical GPUs 8–15 will make each child use physical GPUs 0–7 instead, potentially colliding with another job. Preserve the inherited visibility mapping; cuda:{rank} already indexes that mapped set.

    env.pop("HIP_VISIBLE_DEVICES", None)

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

⚠️ [ruff] <RUF100> _{reported by reviewdog 🐶}
Unused noqa directive (unused: S110)

Suggested change

	except RuntimeError: # noqa: S110
	except RuntimeError:

[github-actions]

⚠️ [ruff] <RUF100> _{reported by reviewdog 🐶}
Unused noqa directive (unused: S110)

Suggested change

	except RuntimeError: # noqa: S110
	except RuntimeError: