[cailmdaley]

Claude (Opus) on behalf of Cail.

A working tomographic n(z) for v2 now has a home in sp_validation. cosmo_val reads it from the catalogue entry like any other n(z) file, and the code that made it can regenerate it from a recipe. This is the stand-in Martin and Cail discussed on #weak-lensing-3x2 (2 Nov). The calibration lives with the shape catalogues, and the photo-z group can later replace the method without changing the file cosmo_val reads.

Using it. SP_v2.0 in cosmo_val/cat_config.yaml is the blinded v2 tomographic run's entry. It points shear.redshift_path at the four-bin n(z) and tomo_bin_col at TOM_BIN_ID. cosmo_val only reads that file; it never runs the calibration or touches photometry. To use the n(z) elsewhere, copy the one text file and point redshift_path at it. On candide it is /n17data/cdaley/unions/analyses/shear_tomographic/nz_v2_full/calibration/nz_tomo.txt (world-readable). The entry is Nibi-only, so it declares host: nibi, and the candide path guard skips entries declared for another host.

Regenerating it. cosmo_val/nz/v2_bound9.yaml records the recipe once. Run these inside the container on an allocation:

python scripts/prepare_nz_photometry.py cosmo_val/nz/v2_bound9.yaml   # attach DR6 ugriz + BPZ to the shear selection
python scripts/calibrate_nz.py cosmo_val/nz/v2_bound9.yaml --workers 16 [--figures DIR] [--rebuild-model]

The method is a SOM direct calibration in the KiDS style, kept deliberately simple:

• 101×101 SOM on the ugriz colours plus r, grouped into 2,000 Ward groups.
• Four DR6 BPZ bins, edges [0.10, 0.37, 0.55, 0.76, 1.20].
• Per-bin gold groups, required to have in-bin spectra and |⟨spec-z⟩ − ⟨BPZ⟩| ≤ max(0.4, 5·NMAD).
• Each spectrum weighted by its group's source shear weight per in-bin calibrator.
• z2 fills objects with no canonical z, as z2 − 0.1105.

The code is src/sp_validation/nz_{photometry,calibration,pipeline,diagnostics}.py.

Result. Per-bin ⟨z⟩ is 0.286 / 0.448 / 0.612 / 0.929. Bins shift by at most 0.006 between the full area and the original 600-tile calibration.

The second z filter carries the coverage. Gold retains 51% of the bound-9 shear weight with z2 and 32% with canonical z alone. Over the full area only 39% of the weight has canonical-z photometry.

The four-bin sum has ⟨z⟩ = 0.566. Published v1.4.6.3 has 0.596; the gap sits in the tail above z ≈ 0.8.

SOM diagnostics: SOM plane, gold masks, BPZ vs spec-z

Limits.

• The calibration uses one field: 11,114 EGS DEEP2/3 spectra from the v1.4.6.3 calibration sample. It models neither field-to-field variance nor spectroscopic-success selection, so the n(z) is fit for validation, null tests and mocks, not parameter constraints.
• The n(z) describes only TOM_BIN_ID > 0, about 58% of n_eff. A non-tomographic run over all objects pairs an all-object sample with a gold-only n(z).
• Source weights come from the 22,200-tile bound-9 selection (191M rows), not the 19,940-tile catalogue cosmo_val measures (170.6M rows).
• Groups empty in the 600-tile reference skip the BPZ-vs-spec-z check.

Checks.

• On candide the module reproduces the delivered products over all 191,016,178 rows: zero TOM_BIN_ID changes and ⟨z⟩ equal to 9e-16, from both the frozen model and --rebuild-model. The rebuilt SOM, groups and gold masks are bit-identical. The gate code is in scratch/cdaley/nz_som_reproduction/.
• New unit tests cover moment-conserving deposition, the z2 rule, gold masking, the writer format, and the cosmo_val reader and SACC NZ round trip.
• CI is green.

Closes #455. #274 (the photo-z group's SOM n(z)) stays open; this is the placeholder it will replace.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JK7o9G4LJE9R9mdaj4rJ4J