Skip to content

UK local solve through the public calibrate front door, increment 2 of #612 - #634

Merged
juaristi22 merged 6 commits into
mainfrom
uk-frame-inc2-local-front-door
Aug 10, 2026
Merged

UK local solve through the public calibrate front door, increment 2 of #612#634
juaristi22 merged 6 commits into
mainfrom
uk-frame-inc2-local-front-door

Conversation

@juaristi22

@juaristi22 juaristi22 commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Increment 2 of #612, on the #618 Frame carrier now on main: the UK local/rowwise solve moves onto the public populace.calibrate front door — the same entry point the US production paths use (build_us_acs_local_release.py, build_us_fiscal_refresh_release.py, the exact-k ladder) — so UK weights take the kernel-enforced WeightKind.CALIBRATED transition and the kernel mints the mass record. All six roadmap commits have landed; draft pending the final CI pass and the compile-cost probe note.

Commit roadmap

  1. Delete the stacked research harness (landed). The pre-ladder areas x households cross-product solve had zero production callers (tests only), was superseded by the rowwise ladder path, and structurally can never route through calibrate() — its weight vector is longer than any entity table. Removed: solve_stacked_local_weights, the stacked doctrine route, local_runner's candidate build path, and the stacked matrix/long-weights helpers in local_geography. Kept: solve_prepared_local_weights + StackedLocalSolveResult (the rowwise path consumes them until commit 3), the reviewed doctrine constants, past_cap_census, and align_area_targets. Everything deleted was UK-specific uk_runtime/ code — nothing in country-agnostic populace-calibrate is touched. The uniform-target-surface check moved beside its only remaining consumer in local_rowwise. sort_households_by_id retires with its only consumer; positional-assignment safety on the surviving paths rides on the Frame kernel's sorted-group-id invariants and validate_uk_national_frame.
  2. Ladder-clone result becomes Frame-native, in memory (landed). Per-entity clone-index columns ({entity}_clone_index) so Frame's column-uniqueness holds; the written H5 keeps the legacy schema (single clone_index per table — the writer renames at the export boundary), asserted by an old-vs-new payload test on synthetic tables.
  3. Rowwise solve through public calibrate() (landed). Area×metric targets as a declarative TargetSet (area membership as filters, Target.metadata carrying area/metric labels), today's solver constants passed explicitly and pinned (epochs 512, lr 0.15, ratio 100.0, cap 10.0), kernel-minted mass record reconciled with the writer's chain-currency fence, past-cap census keys unchanged. The three private populace.calibrate.solve imports die here. A differential test pins the declarative compile to the COO assembly's matvec row for row; the kernel record now declares the realized mass factor (the hand-minted predecessor left it None). Compile-cost probe at real scale: recorded when the next credentialed candidate build runs.
  4. Retire the unfenced dataset: Any surface (landed) (rowwise_dataset narrowing to Frame | str | Path, the candidate tool).
  5. max_weight_ratio anchors differently per arm: 5x vs design (dense) but ~25x effective (sparse refit re-anchors) — declare one stretch contract #493 stretch-contract adjudication record (landed) (no numeric change; revision stays with the UK dense/local epic: rowwise constituency-grain dataset by cloning the national staging pool (US dense-arm analog) #495 increment-6 measured-fit review) + the honest household_weight-drop assessment (blocked by UK carrier swap, increment 1 of #612: carry the national build on populace Frames (stacked on #610 + #617) #618's own carrier contract and One country-agnostic gate battery: phased evaluation, unconditional reporting, per-country gate selection (#578) #611-fenced readers; documented, deferred).
  6. Changelog (landed, extended with each commit).

Fences

  • Touches none of the One country-agnostic gate battery: phased evaluation, unconditional reporting, per-country gate selection (#578) #611-consumer-half gate files (terminal_gates.py, weighted_integrity.py, release_input_coverage.py, national_build.py).
  • Runs in parallel with the increment-3 stage-runtime PR; file-disjoint by construction, and that PR keeps load_uk_national_frame's signature stable for the candidate tool here.
  • firm_generation.py's two private imports are out of scope (follow-up issue; its per-target loss weights and external torch matrix need their own public-seam design).

Refs #612, #578 (item 2.5 ladder), #493, #495.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Adversarial self-review complete; all confirmed findings fixed in 27c34cd. Two independent reviewers attacked the two increments before handing them to a human pass; findings below are ranked, each was confirmed with a repro before fixing, and the clean list is included so the review is falsifiable.

Findings and fixes

  1. BLOCKER — inherited clone_index crashed the write path on production-shaped input. The real staging input's household table carries a candidate-tier clone_index (SPI/pool lineage — spi_support.py's strata columns; the Carry UK build datasets as populace Frames: retire the UKNationalDataset / UKRowwiseDatasetResult shadow types #612 charter's collision census) that no fixture had. The pre-Frame clone silently overwrote it in place; the Frame carrier left it beside the new per-entity household_clone_index, and the writer's rename then produced duplicate column labels — a raw ValueError: cannot reindex on an axis with duplicate labels from inside pandas, after the full solve had already run. Fixed: the clone replaces the inherited column explicitly at entry (the same artifact semantics the pre-Frame writer had, now documented at the drop site), the reserved {entity}_clone_index names fail closed instead of being silently overwritten, and regression tests cover the production input shape on both the in-memory frame and the written H5.
  2. MINOR — flipping _CONSERVE_MASS = True bricked the candidate tool after the solve (the manifest's calibration_mass_change block reads the kernel's free-mass record, which a conserve run never appends). Fixed: refused upfront with the reason, before any work runs.
  3. MINOR — mass_reason was invisible to options consumers. Fixed: recorded in CalibrationResult.options beside mass.
  4. NIT — reported initial_loss drifted ~1e-7 relative (float32 optimizer-trajectory head vs the old float64 recompute). Fixed: recomputed in float64 from the compiled estimates; final_loss was already bitwise identical.

Noted, deliberately not fixed here: calibrate_l0_refit has no mass_reason parameter — that belongs with the problem-level seam design chartered in #636.

Attacked and found clean (falsifiability)

  • Declarative-vs-COO equivalence: the TargetSet compile is bitwise equal to the hand-assembled matrix (dense compare), and the solved weights are bitwise identical to the pre-migration solver on both the plain path (epochs 512, seed 0) and the L0-budget path (target_records=40) — identical n_nonzero, past-cap counts, and diagnostic estimates.
  • ID alignment/dtype coercion refuses loudly ("match the problem"), never realigns silently; unsorted group ids fail closed at Frame construction.
  • Doctrine forwarding pinned by test (epochs 512, lr 0.15, ratio 100.0, cap 10.0, free mass, reason); calibrate's default scale/weight rules match the doctrine's declared rules.
  • Kernel mass/kind semantics verified under free and conserve mass; mass_reason validation complete (conserve+reason, blank, non-str all refused).
  • Writer fences: post-gate geography mutation refused; mass-log currency checked against engine-materialized weights; a mutated household_weight column is superseded by typed weights per the documented "typed weights authoritative" contract.
  • Deletion fallout: zero surviving references to any deleted symbol repo-wide; ladder entry narrowing refuses duck carriers; the crosswalk route unchanged.

🤖 Generated with Claude Code

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strong direction — deleting local_runner/local_solver and routing the rowwise solve through populace.calibrate.calibrate retires the three private imports and the second entry point that owned its own defaults, which was the concrete harm #612 named. The mass_reason addition to calibrate() is a clean, minimal way to keep the domain reason on the kernel-minted record. Notes:

1. result.diagnostics is consumed positionally, and the check that guards it can pass by coincidence. solve_uk_rowwise_weights_under_doctrine builds initial_estimates / final_estimates by iterating result.diagnostics in order and then zips them against problem.target_frame rows. The guard is len(result.diagnostics) == len(problem.target_frame) plus exact equality of the target values. For a local surface, many target values legitimately repeat (zeros, small counts across areas), so a reordering inside calibrate() could satisfy both checks while silently misattributing every diagnostic to the wrong (area, metric) — and the past-cap census and the labelled diagnostics frame are exactly the artifacts that would be wrong. Since you already put area_type/area_code/metric into each Target.metadata, I'd assert alignment on the target name (or that metadata triple) rather than on the value, or index by name instead of by position. Is diagnostics order a documented part of the calibrate() contract? If it is, a one-line comment citing it would do; if it isn't, this is depending on an implementation detail of the front door you just moved behind.

2. np.allclose(compiled_targets, targets_vec, rtol=0, atol=0) is exact equality written obscurely, and it silently returns False for NaN on both sides (which is the right refusal here, but by accident). np.array_equal says what's meant.

3. Rebuilding the finished frame from three tables drops anything else the kernel put on it. You take result.frame.table(...) x3 plus time_period and mass_log into a fresh uk_national_frame(...). Strata and any frame metadata other than time_period on the calibrated frame don't survive that. For today's UK national frame that's probably empty, but it's a lossy reconstruction of a kernel product, and the comment above it ("nothing appended here") reads as if it's preserving the kernel's output rather than rebuilding it. Would result.frame.with_... on the household table be possible instead, so the carrier is only refreshing the persisted household_weight column?

4. The household-mismatch error message is misleading for the common case. frame_ids != problem.household_ids fires on ids or order, but the message only reports len(frame_ids) vs problem.n_households — which will be equal in the ordering case, producing "got 4200 frame rows vs 4200 problem households". Worth naming the first differing id.

5. Retiring min_initial_weight is a real behavior change worth calling out beyond the changelog. Zero base weights are now a hard refusal rather than being floored to 1e-4. I agree with the doctrine ("a dead row must be dropped or revived upstream with a recorded mass change"), but has a real candidate build been run through this to confirm no live path currently feeds zero-weight rows? Same question for UK_LOCAL_MAX_WEIGHT_RATIO — the new adjudication comment (2026-08-07) is a good record, and I read it as explicitly deferring the tightening to #495 increment 6, which seems right.

6. calibrate()'s recorded params dict gains a mass_reason key. Anything downstream asserting the exact key set of that payload (pinned diagnostics JSON, artifact comparisons) will move. Probably covered by the tests, just flagging it as the one outward-facing shape change in an otherwise additive parameter.

7. Nit: _measure is used for both measures and filters (the area mask), which reads oddly at the filter=_measure(area_masks[...]) call site. _constant_vector or similar would say what it does.

Deletion side looks clean — _require_uniform_target_surface moved intact beside its one consumer, past_cap_census kept, and the README/doctrine docstrings were updated rather than left pointing at removed modules. Worth a quick ruff confirmation that nothing in local_geography.py's surviving private helpers (_effective_sample_size, _align_household_frame, _metadata_values, _source_keys) is now orphaned.

juaristi22 and others added 6 commits August 10, 2026 10:07
The pre-ladder stacked areas x households cross-product solve had zero
production callers (tests only), was superseded by the rowwise ladder
path, and structurally can never route through the public calibrate()
front door this increment migrates onto — its weight vector is longer
than any entity table. Removed: solve_stacked_local_weights, the stacked
doctrine route, local_runner's candidate build path, and the stacked
matrix/long-weights helpers in local_geography. Everything deleted was
UK-specific uk_runtime code; country-agnostic populace-calibrate is
untouched.

Kept: solve_prepared_local_weights and StackedLocalSolveResult (the
rowwise path consumes them until the calibrate() migration lands), the
reviewed doctrine constants, past_cap_census, and align_area_targets.
The uniform-target-surface check moved beside its only remaining consumer
in local_rowwise. sort_households_by_id retires with its only consumer:
positional-assignment safety on the surviving paths rides on the Frame
kernel's sorted-group-id invariants and validate_uk_national_frame. The
SPI lineage test now asserts the anti-doubling property directly on
source_household_key instead of through the retired long-weights
representation.

Refs #612, #578.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The rowwise release path's clone result was the last UK shadow carrier on
this arm: three raw DataFrames plus a weight-kind string and a mass log,
kept off Frame only by the artifact's shared clone_index column name on
all three entity tables (Frame's flattening rule refuses it). Clone
indices now land on per-entity {entity}_clone_index columns in memory, so
the result carries a validated UK national frame — typed household
weights under the #501 kind-fence chain, the mass log, and time_period as
frame metadata — and the cross-clone link-alignment fence reads the
per-entity names.

The export boundary is unchanged: the writer materializes engine tables
from the frame (typed weights authoritative) and renames the per-entity
columns back to the artifact's single clone_index name per table, in
place, preserving column order; the ladder-clone test pins the written
schema (column lists, clone_index position, attrs) so a carrier change
can never leak into the H5. The candidate tool asserts the kernel kind
transition explicitly and hard-constructs the calibrated frame with the
refreshed weight column and appended calibration record — the #618 stage
idiom — until the calibrate() front-door migration owns both. The
crosswalk route keeps its table-carrying result; its carrier is a
recorded follow-on.

The pool-lineage fixture now ascends its household ids: every production
producer is a Frame load, which guarantees sorted group ids, and the
carrier lets Frame raise on unsorted input rather than sorting rows
itself (the FRS-2024-25 bug class).

Refs #612, #495.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nt (#612 inc 2)

clone_uk_dataset_with_ladder_geography refuses the duck-typed in-memory
carrier: an in-memory input must be a populace Frame (weight kind and mass
log from the typed weights and frame, nothing defaultable), and only an
H5 path keeps the documented attribute-less DESIGN semantics. The
crosswalk route keeps its duck input — its carrier swap is the recorded
follow-on. Test ducks became real frame constructions; the negative-weight
guard is exercised at the raw-table entry, where a kernel-refused input
cannot mask it.

UK_LOCAL_MAX_WEIGHT_RATIO carries the populace#493 adjudication record:
the bound stays 100.0 (vs the US design bound 5.0, realized 4.994, and
the exact-k ladder's 20.0) because tightening without measured fit would
be an arbitrary cutoff; the revision path remains the #495 increment-6
calibrated-candidate review through this constant and its pinned test.

national_frame's docstring records the household_weight column-drop
assessment: the column is a materialized export contract regenerated by
engine_tables, the carrier's own validation makes it load-bearing, and
the drop is sequenced behind the #611 consumer half and the reader moves.

Refs #612, #493, #495.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inc 2)

The UK local solve was the last consumer of populace.calibrate's private
internals (_optimize, _search_l0_lambda_for_budget,
_torch_constraint_matrix) — it hand-assembled a COO/CSR system because it
had no Frame at the solve site. With the ladder clone Frame-native, the
doctrine solve now expresses the area x metric surface as a declarative
TargetSet (the metric vector as the measure, area membership as the
filter, both closing over the problem's own arrays) and calls the same
public calibrate() entry the US production paths use. The kernel enforces
the CALIBRATED kind transition and mints the calibration mass record;
calibrate() gains an additive mass_reason parameter so a free-mass record
can carry domain context (the bound target families) instead of the
generic reason, and the record now declares the realized factor.

The hand-assembled UKRowwiseLocalMatrix stays as the fail-closed surface
definition and evidence substrate: builder refusals (uncovered areas,
unreachable nonzero targets), the per-area support summary, the labelled
diagnostics frame, and the populace#492 past-cap census all keep their
pre-migration vocabulary — and a differential test pins the declarative
compile to the COO assembly's matvec row for row, so the two
representations provably derive from the same numbers. The doctrine's
constants ride into the front door explicitly and are pinned by a
forwarding test (calibrate's own defaults differ, so default-drift would
change behaviour). solve_prepared_local_weights, StackedLocalSolveResult,
the optimizer floor (zero base weights are refused; tiny positives are
legal seeds), and the hand-minted rowwise_calibration_mass_record retire;
local_solver.py dissolves into local_rowwise.py.

Refs #612, #493, #495.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… precision (#612 inc 2)

The review's blocker: the production staging input's household table
carries a candidate-tier clone_index (SPI/pool lineage) that the synthetic
fixtures never had. The pre-Frame clone silently overwrote it; the Frame
carrier left it beside the new per-entity name, and the writer's rename
then produced duplicate column labels — a raw pandas crash after the full
solve. The clone now replaces the inherited column explicitly at entry
(same artifact semantics as the pre-Frame writer, now documented), and the
reserved {entity}_clone_index names fail closed instead of being silently
overwritten. Regression tests cover both, on the production input shape.

Also from the review: the candidate tool refuses _CONSERVE_MASS upfront
(the manifest's calibration_mass_change block reads the kernel's free-mass
record; flipping the constant used to brick the tool after the solve);
calibrate() records mass_reason in CalibrationResult.options so manifest
consumers see the declared reason, not only the mass log; and the doctrine
solve reports initial_loss in float64 from the compiled estimates again
(the trajectory head is a float32 optimizer value — the reported figure
had drifted at the eighth digit).

Refs #612.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 inc 2)

Vahid's review on #634: diagnostics alignment is asserted per-row by
target name (order is documented only as compiled-problem order, and
values repeat on a local surface, so value equality could pass a
reordering by coincidence); the finished-frame rebuild says it is a
rebuild and refuses to drop kernel strata/metadata instead of losing
them silently; the household-mismatch error names the first differing
id and calls out the same-ids-different-order case; exact target
equality reads np.array_equal; the filter/measure closure factory is
_constant_vector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-frame-inc2-local-front-door branch from 27c34cd to a3de5dd Compare August 10, 2026 08:53
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Thanks — all seven points addressed; fixes are in a3de5dd (the branch is also freshly rebased across the #638 rename, so the whole diff is now in the microcosm.* namespace).

1. Diagnostics alignment is now asserted by name, not assumed. You're right that the value-equality guard could pass by coincidence on a repeating surface — and the order is not a documented contract: CalibrationResult.diagnostics documents alignment "to the compiled problem rows", which is the front door's internal order, not the declared one. The solve now builds the TargetSet once, keeps it, and asserts diagnostic.name == target.row_name positionally across the whole surface before any evidence table is assembled — a reordering inside calibrate() now fails loudly with the first mismatched row named, instead of misattributing the census. Regression test included (a wrapper around the real calibrate that reverses the diagnostics tuple must be refused).

2. np.array_equal it is — same refusal, said plainly (and NaN still refuses, now by documented behavior rather than accident).

3. The rebuild is honest now, and fail-closed on dropped surfaces. There is no Frame.with_table-shaped API (the kernel product is immutable; with_weights is the only carrier), so refreshing the persisted column does require reassembly through uk_national_frame. What changed: the comment now says "rebuilt" instead of implying preservation, and two guards after reassembly refuse to ship if the kernel product carried anything the assembler drops — strata differing from the rebuilt frame's, or metadata beyond time_period. Today both are trivially equal (the UK charter defers strata adoption and the clone frame's metadata is exactly the period), so the guards are dead weight until the day they aren't — which is the day silent loss would have started.

4. The mismatch error now names the first differing id and distinguishes the ordering case: same id set in a different order reports "same households in a different order; first mismatch at row N (frame X vs problem Y)", differing sets report the first divergent id, and only a genuine length difference reports counts.

5. Honest answer: no real candidate build has been run through the zero-weight refusal yet. PR CI is secrets-free and the staging input sits behind the UKDS credential fence, so the confirmation is the recorded post-merge step in the PR body (one credentialed candidate rebuild, diffed against the shipped artifact). What I can say now: the refusal is upfront (before any solve work), names the count and the remedy, and the shipped 2023 staging artifact's household_weight column has no zeros in its committed build-record summary. If the credentialed rebuild does surface a zero-weight row, that's a real upstream bug the old max(base, 1e-4) floor was hiding — which is the argument for the refusal. On UK_LOCAL_MAX_WEIGHT_RATIO: yes, the adjudication comment defers tightening to the #495 increment-6 measured-fit review; the doctrine keeps refusing caller overrides in the meantime.

6. Checked: nothing pins the options key set. The only shape assertion on CalibrationResult.options in the suite is the warm_start_weights payload test; the pinned-diagnostics artifacts compare specific keys, not the exact set. The new mass_reason key is covered by its own assertions in test_solve.py.

7. Renamed — the closure factory is _constant_vector now, so filter=_constant_vector(area_masks[...]) reads as what it does.

On the orphan sweep: the four local_geography privates you flagged (_effective_sample_size, _align_household_frame, _metadata_values, _source_keys) were deleted with the stacked route in the first commit — a repo-wide grep finds no definitions and no references on the branch, so nothing is orphaned.

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed a3de5dd0. Everything is addressed, and the diagnostics fix is better than what I asked for.

  • Name-aligned diagnostics. I suggested asserting on the area_type/area_code/metric metadata triple; asserting diagnostic.name == target.row_name is cleaner and I checked it holds by construction — matrix.py builds the row names from target.row_name ("name@period"), which is the same string TargetDiagnostic.name documents. Two things fall out that are worth noting because they make the guard stronger than a spot check: zip(..., strict=True) means a length drift can't slip past even if the earlier count check is ever relaxed, and because TargetSet already rejects duplicate (name, period) keys, the target names (area_type/area_code/metric) now give you a second, independent enforcement of the same uniqueness _require_uniform_target_surface demands. The dataclasses.replace(result, diagnostics=reversed(...)) test is the right way to prove it — it fails the name check while passing both the length check and, on a symmetric surface, value equality.
  • Lossless-or-refuse rebuild. This is the better answer to what I raised. I was asking whether the drop was intentional; making the rebuild say it's a rebuild and then refuse to ship if the kernel product held strata or metadata the assembler can't carry converts a silent loss into a boundary marker for the day it stops being trivially empty. I checked the happy path doesn't now hard-fail: finished is assembled from result.frame's own person table, so the default strata Series is identical on both sides, and time_period is idempotent through uk_time_perioduk_national_frame, so the metadata dicts compare equal.
  • Household-mismatch error now distinguishes same-ids-different-order from different-id-sets and names the first differing row; np.array_equal; _constant_vector. All as asked.

Nothing further from me. My remaining questions from the first pass were about operational confirmation rather than code — whether a real candidate build has been through the no-more-min_initial_weight path without hitting a zero base weight, and whether anything downstream pins the exact key set of calibrate()'s recorded params (which gains mass_reason). Neither is a code change; worth a sanity run before this goes out rather than a diff.

Approving. Test jobs still pending at the time of writing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants