Add Layer Norm Backward challenge (medium #144)

[programmingGod-byte]

Summary

Adds Layer Normalization Backward (Medium #144) to complete the transformer normalization suite alongside #113 (LayerNorm forward), #142 (RMSNorm backward), and #143 (Fused Residual Add + LayerNorm with saved statistics).

Challenge Overview

• Operation: Given x, gamma, saved mean, rstd, and upstream gradient dy of shape (N, C), computes the input gradient dx of shape (N, C) and parameter gradients dgamma and dbeta of shape (C,).
• GPU Concepts Tested:
  • Row-wise reductions across feature channels $C$ to compute mean projections for $dx$.
  • Column-wise reductions down the sequence/batch dimension $N$ for accumulating $d\gamma$ and $d\beta$.
• Benchmark Size: Measured with $N = 16,384, C = 4,096$ (~800 MB footprint, well within Tesla T4 VRAM budget).

Test Coverage

Includes 11 targeted test cases:

• Hand-computed analytical example checking reference implementation correctness.
• Edge cases ($1\times 1$, single row $1\times 3$, uniform rows with zero variance).
• Zero upstream gradient verification.
• Negative scale with correlated inputs.
• Ragged tails ($N = 4,099$) to catch multi-block reduction accumulation bugs.
• Realistic LLM hidden size ($N = 1,024, C = 2,048$).
• Outputs pre-filled with 123.0 to ensure uninitialized atomic accumulation fails.

Files Included

• challenges/medium/144_layer_norm_backward/challenge.html
• challenges/medium/144_layer_norm_backward/challenge.py
• All 6 starter templates: .cu, .triton.py, .pytorch.py, .jax.py, .cute.py, .mojo

Validation

• python scripts/validate_challenges.py: 0 errors, 0 warnings
• All code formatted with black and clang-format.

[kunal-mansukhani]

Thanks @programmingGod-byte! Verified the reference against float64 autograd on every test (max error 2.6e-3 on the 16384x4096 perf case, well inside 1e-3 rtol), and lint, validation and Mojo 1.1 format are all clean. Merged as medium #144, and it's live at https://leetgpu.com/challenges/layer-norm-backward. Nice addition to the normalization set.

[claude]

Reviewed against the CLAUDE.md checklist. Everything passes: the HTML structure, the example matches generate_example_test() (I checked the values by hand), the perf bullet, the starters, the reference impl, and memory use (about 0.8 GB per test, well under 1/5 of 16 GB). A reference CUDA solution passed the example, all functional tests and the performance test on a T4.

There was one small checklist gap: the test comments didn't say which mistake each case catches. I've fixed that in #354, which changes comments only. Looks good to merge with that.

🤖 Generated with Claude Code