feat(minimax-h3): single-file pruned/int8-convrot transformer support (PR 6)

[lstein]

Summary

PR 6 of the MiniMax H3 stack (on top of #144): load Comfy-Org single-file H3 transformers — including the pruned int8-convrot fl2va (19.5 GiB) — as a checkpoint-format main model that overrides the transformer while the diffusers folder keeps providing the text encoder, tokenizer/processor, and both VAEs.

Why: the bf16 folder transformer is 62 GiB — on a 48 GB card only ~53% stays resident and every denoise step streams ~30 GiB from RAM. The pruned int8 file is fully resident with room to spare, and has ~40% fewer FLOPs (the AdaLN branches are replaced by a precomputed rank-8 time-embedding curve).

Format support (semantics verified against Comfy-Org/comfy-quants, comfy-kitchen, and ComfyUI's model code — reimplemented, not copied)

• int8_tensorwise: per-output-channel symmetric int8 + fp32 scales, declared by <layer>.comfy_quant JSON markers. Weights stay int8 in VRAM (Int8ConvrotLinear buffers); dequant happens per forward in the compute dtype (~620 MB transient at the largest layer).
• convrot: grouped-256 regular-Hadamard rotation folded into the stored weights. The Hadamard is symmetric orthonormal, so derotation is the same grouped matmul — no runtime activation rotation. Roundtrip pinned by tests against a reference quantizer.
• Pruned "AdaLN curves": adaln_t_table [1025, 8] lerped at t∈[0,1] replaces the time embedder; rank-8 AdaLN projections, no silu, fp32 modulation. Implemented as a subclass of the vendored transformer (vendored file untouched). Timestep-domain equivalence with ComfyUI verified in review (t = 1−σ, keyframe pin 0.999, unique-t dedup).
• The full bf16 and pruned bf16 single files load through the same path (time-embedder renames verified against the real HF file header via range request; the no-marker path skips the int8 swap).
• fp8-scaled files are rejected early (header-only marker check) with a clear unsupported-quantization error. GGUF: out of scope. ref2va files are rejected by filename (keys are indistinguishable from fl2va).

Wiring

• Main_Checkpoint_MiniMaxH3_Config (fingerprint: audio_patch_proj.weight + video_patch_proj.weight + blocks.0.attn.qkv_proj.weight — checked for collisions against every other family's single-file fingerprint, both directions); identification fixture included.
• The checkpoint loader serves ONLY SubModelType.Transformer (Wan-GGUF precedent) and errors helpfully for anything else; shape-inferred config (verified key-for-key/shape-for-shape against the real 19.5 GiB header).
• minimax_h3_model_loader v1.1.0: optional transformer_model field (checkpoint-format H3 mains); default workflows' pinned versions bumped.
• Linear UI: "Transformer (single file)" combobox in Generation Settings → Advanced (gated on H3); checkpoint mains hidden from the main model picker; metadata recall for the override (skips silently if the model was uninstalled); params persist v5→v6 with migration fixtures.

Known limitations (review-documented)

• Int8 layers are not autocast-wrapped: residency is all-or-nothing, so the single file needs ~20 GiB free VRAM. Graceful partial-load degradation remains the folder model's job.
• ⚠ Version-collision ledger: upstream main is still params _version 4; this stack claims v5 (PR 4) and v6 (this PR), and the flux2 branch also claims v5 — whichever lands second renumbers.
• A pre-existing global gap (all estimators): user-set max_cache_vram_gb bypasses working-memory reservations.

Testing

• Backend: H3 suite 40 tests + identification (incl. new fixture) + workflow-registry — 144 passed; ruff (CI-pinned 0.11.2) clean.
• Frontend: tsc / eslint / prettier / knip clean; 1720 tests passed; vite build OK. Schema regen committed (node field).
• Adversarial fresh-context review completed; all findings fixed (full-file claim-then-crash, early fp8 rejection, dequant transient, modulation dtype, recall handler) or documented (all-or-nothing residency).
• Manual gate: install the single file by direct path (probes as minimax-h3 / checkpoint / fl2va, pruned), select it under Advanced → Transformer (single file), run t2v; expect ~31 GiB VRAM steady state and no per-step streaming.

🤖 Generated with Claude Code

[lstein]

Critical fix amended in 1f1a342a2e: the fused SwiGLU input projection (mlp.fc1 → ff.net.0.proj) has its gate/value halves in the OPPOSITE order in the remote-code layout vs diffusers' SwiGLU — [gate; value] (silu on the first half) vs [value; gate] (silu on the second). The converter passed it through unswapped, so every block's MLP computed silu(value)·gate and generations came out as structured noise (both user runs reproduced this; latents were sane-scale but spatially uncorrelated).

Found by layer-wise A/B against the diffusers folder as ground truth on identical production-built packed inputs: first divergence at the token refiner (cos 0.43 after a bit-exact context_embedder), then bit-exact half-crossover proof (file[:H] == folder[H:] and vice versa). After the swap (weights + per-row scales of quantized fc1s): full-forward video cos 0.99985 / audio 0.99976 vs the bf16 folder model, residual ≈ the int8 quantization floor.

Why nothing caught it: key/shape dry-runs can't see intra-tensor layout; the ground-truth weight audit sampled qkv (order verified) and fc2, but fc1 — the one other tensor with internal semantic layout — wasn't sampled, and no unit test pinned its content. A regression test with distinguishable halves now does, and the lesson is recorded: every FUSED tensor family in a converter needs a content-level ground-truth check, not just shape.