Release v1.8.0 — reconcile point-broadcast + notebooks + frames-family hardening#191
Merged
Conversation
…ssets (#141) * docs(reconcile): close out Epic 11 cutover (#138) + durable de-risk assets The cross-repo cutover is already complete (verified read-only): C1 repointed views-models's reconciler_factory to views_frames_reconcile (vm#191, PR #202); C2 deleted views_postprocessing/reconciliation/ (vpp#62, PR #63); pipeline-core's cycle is already broken by the DIP port (#195/#217), so #221 is an optional cosmetic tidy. #138 was a stale-open tracker the views-frames checklist never recorded. This closeout adds the durable views-frames-side assets (no src/ change; frozen v1 surface untouched; CONFORMANCE_FLOOR stays 1.0.0): - scripts/verify_reconcile_parity.py — standalone parity verifier. --oracle re-runs views_frames_reconcile against the frozen views-reporting torch-oracle fixtures and reports drift (PASS: 0.000e+00 abs/rel vs oracle); --compare diffs two reconciled parquet outputs on a real production slice. A re-runnable record of the cutover's safety chain (new == vpp == torch oracle) and a tool for future cutovers. - docs/guides/reconciliation_migration_and_cutover_runbook.md — procedural template (mirrors publishing-to-pypi.md): the release→repoint→verify→delete order, the verification gates, rollback; the Epic 11 instance marked COMPLETE with real PRs. - reports/postmortems/2026-06_reconciliation_cutover_epic11.md — the cutover closeout (companion to the build postmortem): the bit-identity chain, the phased PRs, and the stale-tracking lesson. - register C-58 (Tier 3, open): the cutover was verified against the oracle, not a live production slice; --compare is the named mitigation. Header 55/10/45/10. (The expert review's "batched-deploy" concern did not materialize — the cutover was phased.) - pyproject: broaden the scripts/ E501 per-file-ignore to all dev scripts. ruff + validate_docs + register-parse + pytest green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(reconcile): address /review findings on the cutover closeout (#138) Review of PR #141 surfaced three real findings (all in the new dev tool + runbook); fixed and verified: - scripts/verify_reconcile_parity.py: the verifier could report a vacuous PASS. --oracle now FAILs loudly on an empty parity/e2e fixture (instead of returning True over a skipped loop); --compare now FAILs when the two reconciled outputs share no matched rows or no comparable numeric columns (an empty inner-join is a verification failure, not agreement — silently passing it would defeat the tool). - docs/guides/reconciliation_migration_and_cutover_runbook.md: the Phase 4 / gate-table "grep is empty after Phase 4" claim was false and self-contradicted the postmortem (transitional importorskip + docstring references legitimately linger). Corrected to "no live import remains; importorskip/prose may persist — sweep as doc hygiene". Verified: ruff clean; --oracle PASS; --compare disjoint -> FAIL, matching -> PASS; pytest green (2 expected skips). /review-diff verdict: CLEAN. No src/ change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
One full postmortem per sister effort (companions to the existing tower + cutover postmortems): the leaf data contract (views_frames), the summarize sibling (views_frames_summarize), and the reconcile sibling (views_frames_reconcile) — why/what/learned/what-we'd-do-differently. Docs only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…vernance (#147) Side-quest: a `notebooks/` showcase folder (synthetic-data, public-API-only, self-contained, un-gated — like research/ and scripts/). Three roadmap-only notebooks (markdown plans, no demo code yet) + a README: - 01_frames.ipynb — the frames as a typed storage contract. - 02_summaries.ipynb — posterior summaries over a distribution zoo. - 03_reconciliation.ipynb — grid->country reconciliation, with plots. Each carries a section-by-section roadmap, the open questions, the conventions, and a "panel additions" cell folding in an expert-method-review wishlist (calibration/coverage, ET-vs-HDI + mean/median, sample-size sensitivity, failure-mode section, known-truth recovery, a toy-lattice map view, decision relevance; reconciliation-literature context + bit-identity != method-quality + a does-it-help check). One per package; all three frames live in 01 (the sibling family). Register (reports/technical_risk_register.md): - D-11 (disagreement, settled) — frame serialization-convenience placement: pandas never on the frozen leaf; parquet via a format-selectable save (B') only if/when wanted; transitional/removable scaffolding stays off the leaf. Extends the C-52 accretion cross-ref. From an /expert-code-review. - C-59/C-60/C-61 (Tier 3, open) — notebook methodological-completeness gaps from an /expert-method-review: intervals shown without calibration/coverage (C-59); bit-identity risks reading as method validation (C-60); no spatial/map view (C-61). Mitigations folded into the roadmaps. Header 58/13/45/11 (reconciles). /review-diff: REVIEW -> fixed (ambiguous C-37 refs qualified as views-postprocessing C-37; severe_scenario -> expected_shortfall in nb 03). Gate green: ruff, validate_docs, register-parse, pytest. No src/ change; CONFORMANCE_FLOOR 1.0.0. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…es / reconciliation) (#174) Build notebooks/ from roadmaps into three runnable, public-frozen-API-only, synthetic-data showcases (01 frames / 02 summaries / 03 reconciliation) + shared infra (_synthetic.py ground-truth generators, [docs] extra, non-blocking nbmake CI). Executes notebook epics #148/#156/#166 incl. #151. No src/ change; CONFORMANCE_FLOOR 1.0.0. Includes the /review + /review-diff fixes (CI nbmake path, analytic mode truth, narrowed C-32 prose).
) Closes out the three notebook-completeness risks now that the showcase notebooks (#174) are merged. Header counts open 13→10, resolved 45→48. Docs-only.
… design ADR (#176) Epic #142. S1 #143 native point-country broadcast (ADR-023 amend, 1.7.0->1.8.0); S2 #144 self-describing mode via reconcile_result/ReconciliationResult (returned, not stamped on the leaf — D-12); S3 #145 ADR-024 principled-reconciliation design + C-37 docstring fix (C-62). Equal-count path byte-exact (oracle 0.000e+00); leaf untouched; CONFORMANCE_FLOOR 1.0.0. Per-issue rituals + final /review (one docstring-precision fix). No publish.
…/60/61 relocation (#177) Register C-63 (Tier 2, Open) from the repo-assimilation pass; relocate the resolved C-59/60/61 into the Resolved section. Docs-only; header 60/12/48/12.
… + review-rr curation (#178) C-64 (reconcile has no CIC) + C-65 (non-finite blocked-path) from /test-review; C-63 extended (contract+test side); /review-rr strategic curation (narrative 3->14, clusters refreshed, D-04/05/06 resolved-by-events). Header 62/14/48/12. Docs-only.
…ses C-64, #180) (#187) The reconcile package (Epic 11, v1.7.0; extended v1.8.0) shipped without a CIC, and Summarize.md §2 explicitly fenced reconciliation out — ADR-006 requires intent contracts for non-trivial classes. Author docs/CICs/Reconcile.md as a package-level CIC (§1–§11, modeled on Summarize.md): the sum-to-country / zero-preservation / non-negativity / de-mutation guarantees, the point/aligned mode contract, the five fail-loud validation guards + the per-draw-approximation caveat, and the green/beige/ red test alignment. List it under Active Contracts in docs/CICs/README.md. Resolves register C-64 (relocated to Resolved; open 14→13). Docs-only — no src/ or test change; validate_docs + ruff + pytest 100% green. Epic #179 / S1. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…s C-63, #181) (#188) The leaf advertised "immutable value objects", but only the index is setflags(write=False)-enforced; the value buffer is writeable (index.py:55-56 vs the frame constructors' bare self._values = values), so an in-place .values mutation silently corrupts buffer-sharing frames. Enforcing it via setflags would be "tightening an invariant" on the frozen-surface `values` — a MAJOR under GOVERNANCE/ADR-018, triggering the cross-repo coordinated bump — disproportionate for a hole the audit found nothing exercises. Correct the contract instead of enforcing it: ADR-025 records that value-buffer immutability is by convention (writeable on purpose, to preserve zero-copy / mmap; mutating .values is unsupported) and the index is the enforced one, and records the setflags-enforce as a deferred MAJOR-rider. Reword the three frame CICs §9/§3 + README design principle 3 to match; correct the principle cross-ref number (2→3). Resolves register C-63 (relocated to Resolved; open 13→12). Docs-only — no src/ or test change; validate_docs + ruff + pytest 100% green. Epic #179 / S2. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
) (#189) Add the direct adversarial tests the 100%-coverage suite masked (coverage ≠ contract-failure testing); no src/ change, 100% line+branch coverage held: - C-65: non-finite (NaN / ±inf) fail-loud in exceedance/expected_shortfall is now pinned on the BLOCKED (multi-block) path — the bad draw forced into a non-first block via block_rows with block 0 all-finite, so a guard that inspected only the first block would regress silently. Mutation-tested: neutering the guard fails all six cases. - Conformance-suite negatives: assert_reconcile_contract and assert_summarizer_contract are shown to REJECT a deliberately non-conforming implementation (monkeypatched), the leaf's C-51 envelope-negative pattern extended to the sibling packages. - Reconcile mode-corners: reconcile_result.mode for both-points and pre-tiled-cm (both ALIGNED_DRAWS); ReconciliationResult frozen-ness (FrozenInstanceError). Resolves register C-65 (relocated to Resolved; open 12→11). Epic #179 / S3. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The 1.8.0 entry covered only the reconcile right-home work. Complete it for the whole release: the three showcase notebooks (Epic #166) and the Epic #179 hardening pass — Documentation (Reconcile.md CIC #180, ADR-025 immutability-by-convention #181) and Tests (the adversarial red-test batch #182). Set the release date (2026-06-28). Version stays 1.8.0; CONFORMANCE_FLOOR stays 1.0.0 — all additive, no src/ change. Epic #179 / S4. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promote
development→mainfor the v1.8.0 release. Trial-merge is clean (no conflicts);the merged tree is byte-identical to
development. Promote with a MERGE COMMIT, not squash(
maincarries squash "Release" commits + merge commitsdevelopmentlacks — a squash would fightthe histories; dev/main-divergence rule).
What ships in v1.8.0 (all additive;
CONFORMANCE_FLOORstays 1.0.0; not a MAJOR)reconcile_resultmode (S2 — Self-describing reconciliation mode in the returned frame metadata #144), ADR-024 principled-reconciliation design (S3 — [research] Principled joint probabilistic reconciliation — design + ADR (C-37) #145).03_reconciliation.ipynb: reconciling grid forecasts to country totals #166): frames / summaries / reconciliation.docs/CICs/Reconcile.md— package CIC forviews_frames_reconcile(C-64, [S1] Reconcile.md CIC — give views_frames_reconcile a contract (closes C-64, #180) #187).State
developmentgate green: ruff ✅ · mypy ✅ · pytest 100% (line+branch) ✅ · validate_docs ✅.src/behaviour change in the hardening epic.After this merges and
mainCI is green, the v1.8.0 tag + PyPI publish (S6 #185) is triggered separately(
gh release create v1.8.0 --target main).🤖 Generated with Claude Code