Skip to content

fix(unroll): fold const-generic loop bounds - #1569

Open
praveenperera wants to merge 1 commit into
NVIDIA:mainfrom
praveenperera:fix/unroll-const-bound
Open

praveenperera wants to merge 1 commit into
NVIDIA:mainfrom
praveenperera:fix/unroll-const-bound

Conversation

@praveenperera

Copy link
Copy Markdown
Contributor

Closes #1494.

The problem

#[unroll] on a loop bounded by an expression of a const generic, such as while j < N / 8, warned that the trip count was "only known at runtime" and left the loop rolled, so a register array indexed by j moved to local memory.

After monomorphization the bound reaches the unroll pass as mir.div of two mir.constants, not as one constant. induction::const_i128 reads only a direct mir.constant, so the trip count was unknown. rustc's GVN also computes a repeated N / 8 once, at the use that dominates the others. In a nest, that is the outer loop's header, and the inner loops read it from there. So even with the division folded, the outer loop was rejected with "the header computes a value the body reads".

The fix

All changes are in mir-transforms:

  1. unroll_annotated_loops runs sccp on each annotated function once before analyzing its loops. It already ran after unrolling to clean up. The folders in dialect-mir/src/const_fold.rs apply Rust semantics: wrapping add/sub/mul, and no folding of division by zero or INT_MIN / -1.
  2. const_i128 also reads the builtin.constant that sccp materializes.
  3. hoist_header_constants moves constants defined in a loop header into its preheader before the shape checks. The preheader is the header's only entry from outside the loop, so it dominates every use. This handles a folded bound that the body or later code reads, and a constant that sccp materializes for a constant header block argument.

The folders cover integer arithmetic, bitwise operations, shifts and comparisons. Casts and overflow-checked arithmetic in a bound are not yet folded, and the docs say so.

Testing

On an RTX 5090 (sm_120), CUDA 13.1:

  • mir-transforms: two new end-to-end unroll tests on the GVN shape, with one mir.div in the outer header read by both exit tests and by the exit block. One test fully unrolls both loops and checks the folded result. The other unrolls only the outer loop and runs the result. Both fail without the fix, and the outer-only test also fails with just the hoist disabled.
  • unroll_smoke: a new const_generic_bound::<256> kernel. It is nested_lane_range with every bound written LANES / 16, and the host checks it against the same expected sums. verify-code-shape.sh now requires that it uses no local memory. Before the fix, its PTX had 34 ld.local/st.local. After the fix it has none, and the entry is the same size as nested_lane_range (8249 vs 8250 lines). The PTX of the example's other 22 kernels is unchanged byte for byte.
  • The issue's reproducer instantiated with N = 128: no local memory and no branches, down from 18 ld/st.local and 2 branches. It has 64 ld.global and 16 st.global.
  • just check, plus scripts/smoketest.sh in GPU and --compile-only modes: 240/240 each. --full-debug: 234/240. The other six fail the same way on unmodified main: llvm-dwarfdump is missing on this machine, IKET does not support full debug, and libNVVM rejects four with "invalid input program". Full-debug builds skip the unroll pass.

A loop bound written `N / 8` with a const generic `N` reaches the unroll
pass as a division of two constants, so `#[unroll]` reported a runtime
trip count and left the loop rolled. rustc's GVN also computes the shared
bound once in an outer loop's header, which made that loop look like it
computed a value its body reads.

Run SCCP on annotated functions before analyzing their loops, read the
`builtin.constant` it materializes as a constant bound, and hoist header
constants into the preheader before the shape checks.

Closes NVIDIA#1494.

Signed-off-by: Praveen Perera <me@praveenperera.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 20, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@praveenperera
praveenperera marked this pull request as ready for review October 20, 2026 14:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#[unroll] rejects a loop bound computed from a const generic (N / 8)

1 participant