portfolio: fix --save-samples 0-row export (cumsum linear weights + -inf guard) - #35
portfolio: fix --save-samples 0-row export (cumsum linear weights + -inf guard)#35oshaughnessy-junior wants to merge 1 commit into
Conversation
…inf guard) Step 2 of the _rvs cleanup in mcsamplerPortfolio.py paired the sorted index with the LOG-weight ln_wt[k] and cumsummed that, so the "cumulative weight" was never a cumulative probability. Any -inf ln_wt entry (rejected/underflowed sample) poisoned the cumsum (cum_sum[-1] = -inf -> nan on normalize), and the cum_sum > deltaP mask kept 0 rows -- 0 samples exported even at n_eff~9. Fix: pair/cumsum the LINEAR weight wt[k] (already computed one line earlier), matching the canonical mcsampler/mcsamplerEnsemble truncation, and map any non-finite log-weight to -inf (-> zero linear weight) with a degenerate fallback that keeps all rows if no finite-weight sample survives. Verified to reproduce canonical linear-weight index selection exactly on finite peaked input and to keep all real samples (dropping only zero-weight rejects) when -inf entries are present. Change is confined to the portfolio Step 2 block; AV/GMM standalone samplers are untouched. Also document (integrate_likelihood_extrinsic_batchmode) that the sparse sim_inspiral XML from --save-samples carries lnL only and must not be reweighted by likelihood for a weighted-posterior/shape check; point users to the ASCII --extrinsic-proposal-output (full log-weight) / --calibration-export-posterior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tory; add ensemble+pooling harness Correcting my own 3-seed 'cap8 is reliably modest' claim (survivorship bias): 10 cap8 draws span 1->70 with ~40% collapsed to ~1 -- statistically the same bimodal lottery as cap16. correlate-all is strictly worse (collapses 2/3, lnZ 11 nats low). So no single-run config is a posterior on this high-SNR best-fit point; the robust recipe is MANY copies pooled by reliability (the reviewer's original point, now on data). Harness: bench_onsource_ensemble.sh (portfolio must be selected explicitly -- the driver default adaptive_cartesian_gpu is NOT the portfolio/AV) + compare_extrinsic_breadcrumbs.py (weight-correct per-group posterior via --extrinsic-proposal-output; reliability-weighted vs naive pooling; collapse detector = out-of-bounds degenerate GMM). Weight-correct export unblocked by PR #35 (cherry-picked). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Auto-review: not ready to merge while CI is red. The three failed jobs ( Local review: clean merge with |
|
Integrator-gate update: this PR cannot yet satisfy the mandatory portfolio shape gate. The candidate run reports Please merge or depend on the #33 isolation fix, then rerun the standard CPU base-vs-candidate gate (with portfolio strict) and rerun CI. Until then, the red CI plus invalid portfolio gate block merge. |
|
Closing as stale/already upstream. |
Problem
mcsamplerPortfolio.py --save-samplesexported 0 sample rows for peaked (low-to-moderate n_eff) portfolio runs — even at n_eff≈9.Step 2 of the
_rvscleanup paired the sorted index with the log-weightln_wt[k]and cumsummed that, so the "cumulative weight" was never a cumulative probability. Any-infln_wtentry (rejected/underflowed sample) poisoned the cumsum (cum_sum[-1] = -inf→nanon normalize), and thecum_sum > deltaPmask kept 0 rows. The block was inherited from the originalccd70ff7draft (copied from the ensemble pattern) but dropped the linear-weight step.Fix
wt[k](already computed one line earlier), matching the canonicalmcsampler/mcsamplerEnsembletruncation.-inf(→ zero linear weight), with a degenerate fallback that keeps all rows if no finite-weight sample survives.Verified numerically: reproduces canonical linear-weight index selection exactly on finite peaked input, and keeps all real samples (dropping only zero-weight rejects) when
-infentries are present (old code kept 0).Also (docs only, no XML schema change)
The sparse
sim_inspiralXML from--save-samplescarries lnL only and cannot be reweighted by likelihood for a weighted-posterior/shape check. Added a code comment + updated--save-sampleshelp text pointing users to the ASCII--extrinsic-proposal-output(full log-weight) /--calibration-export-posterior.Validation note
The shape-recovery merge gate (S250114ax best-fit point,
--save-samples --save-P 0) needs GPU + data and was not run in the authoring environment — please run it before merge.🤖 Generated with Claude Code