test: posterior shape-recovery merge gate for MC integrators (expensive_before_merging)

[oshaughnessy-junior]

Posterior shape-recovery merge gate for the MC integrators

Adds test/expensive_before_merging/integrators/ — the strong, slow validation suite run before confirming a merge into a production line (not per-commit CI). Motivation: the fast CI gate checks only the integral on one 3-D Gaussian; integrals are easy, while the recovered posterior shape (the weighted cloud CIP/fairdraws consume) can be confidently wrong with a perfect evidence. This suite already caught, in its first deployment: a GMM run with evidence correct to +0.009 nats but marginal JS = 0.29 and widths 2.8–3.8× too broad; the portfolio aggregation bias fixed by #26 (−2.7..−3.1 nats → ≤ +0.12, quantified); and the rift_O4c→rift_O4d GMM efficiency collapse (n_eff 3047→1 at d=6, bisected to an import-time cupy binding with silently swallowed refit failures).

Contents

• shape_recovery.py — self-contained harness (runs against ANY checkout via PYTHONPATH, incl. rift_O4c): seeded random Gaussian-mixture targets (d ∈ {2,4,6,8} × ncomp ∈ {1,3} × 3 seeds, Wishart covariances, FinerNet multigauss recipe), truth = 10⁶ exact rejection fair-draws, samplers exercised through the production add_parameter/integrate[_log]/_rvs API. Metrics: per-dim JS vs a self-calibrating matched-n_ESS floor, mean pulls, width ratios, correlation recovery, lnZ bias. Verdicts: PASS / FAIL / STARVED (starved = n_eff below testability; non-blocking absolutely, gates differentially).
• compare_shape_results.py — differential PR gate: PASS→FAIL/STARVED or metric-worsening on a strict sampler (default AV,GMM) blocks; fail-on-both is reported as pre-existing.
• run_shape_recovery.sh, pytest wrapper (RIFT_RUN_EXPENSIVE=1), READMEs with the merge workflow.
• Breadcrumbs for future developers/agents: AGENTS.md section, RIFT/integrators/TESTING.md colocated with the integrator sources, pointers in test/README.md and test/integrators/README_benchmark.md.

Validation

Standard preset (~10 min/branch, CPU-only, ldas-pcdev11): rift_O4d base, PRs #26/#27/#28, rift_O4c back-check. Results, comparison tables, and the GMM-regression bisect provenance are archived in the RIFT_roboto_paper repo under analyses/integrator_shape_gate/; methodology documented in the paper appendix ap:shape_gate. First-deployment evidence on #26: #26 (comment)

Notes

• CPU-only by design (CUDA_VISIBLE_DEVICES=""): deterministic, and deliberately exercises the cupy-installed-but-no-GPU worker configuration that produced the silent GMM failure. The suite works around the GMM stack's module-level cupy selection via _force_cpu_modules(); the underlying no-device-probe bug is being fixed separately.
• Head nodes have RLIMIT_NPROC=500 — run one suite at a time (SHAPE_JOBS=10-12, OMP_NUM_THREADS=1).

🤖 Generated with Claude Code

[oshaughnessy-junior]

Auto-review completed the required posterior shape-recovery merge gate. I ran the standard CPU-only 96-run matrix on both the refreshed rift_O4d base and this candidate, then used the supplied differential comparator. Result: # blocking regressions (strict=['AV', 'GMM']): 0.\n\nThe quick pytest wrapper also passed (4 passed), and merge simulation, whitespace validation, and touched-Python compilation passed. Existing portfolio failures were reported as pre-existing/warn-only by the suite policy, not candidate regressions.\n\nThis gate is now recorded as mandatory for every integrator-affecting PR in periodic reviews across RIFT repositories.