From 09c4c4e4fd3f64af7aea387c545d1df3e6ce9a1a Mon Sep 17 00:00:00 2001 From: Polichinl Date: Sat, 27 Jun 2026 19:11:24 +0200 Subject: [PATCH] =?UTF-8?q?docs(adr):=20S4=20=E2=80=94=20truth-up=20the=20?= =?UTF-8?q?foundational=20ADRs=20(#74)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per the S1 supersession policy: - ADR-001 (ontology): status -> Superseded by ADR-012, with a banner; original text kept intact as the historical record (it enshrined the deleted shapefile/spatial engine as the stable core). - ADR-012 (new): the current ontology — delivery invariants, the representation seam, the lookup enrichment, the thin pipeline-core-subclass manager, producer data-facts; explicitly excludes the removed spatial engine and downstream statistics. - ADR-002 (topology): amended in place — the acyclic/downward decision stands, but its illustrative layering referenced deleted layers. Appended an Amendment with the cross-repo topology (datafactory->pipeline-core->vpp->faoapi; pipeline-core does not import vpp) and the current internal layering; original text preserved. - ADR index: ADR-001 marked superseded, ADR-002 amended, ADR-012 added. All internal links resolve. Docs-only; ruff clean, suite green. Co-Authored-By: Claude Opus 4.8 --- .../001_ontology_of_views_postprocessing.md | 11 ++- .../ADRs/002_topology_and_dependency_rules.md | 64 +++++++++++++- docs/ADRs/012_revised_ontology.md | 83 +++++++++++++++++++ docs/ADRs/README.md | 11 ++- 4 files changed, 165 insertions(+), 4 deletions(-) create mode 100644 docs/ADRs/012_revised_ontology.md diff --git a/docs/ADRs/001_ontology_of_views_postprocessing.md b/docs/ADRs/001_ontology_of_views_postprocessing.md index 4f386a0..0a32b2e 100644 --- a/docs/ADRs/001_ontology_of_views_postprocessing.md +++ b/docs/ADRs/001_ontology_of_views_postprocessing.md @@ -1,12 +1,21 @@ # ADR-001: Ontology of views-postprocessing -**Status:** Accepted +**Status:** Superseded by [ADR-012](012_revised_ontology.md) (2026-06-27) **Date:** 2026-06-02 **Deciders:** Project maintainers (PRIO MD&D Team) --- +> ⚠️ **Superseded.** This ADR's ontology was built around a runtime spatial-mapping +> engine and bundled shapefiles as the authoritative, stable core. That architecture was +> removed (ADR-011 / C-39): enrichment is now a precomputed GAUL lookup-join and the repo +> is a post-forecast delivery + input-integrity layer. The **current** ontology is +> [ADR-012](012_revised_ontology.md). This document is kept intact as the historical record +> of the original design. + +--- + ## Context The views-postprocessing repository bridges conflict prediction outputs from the VIEWS platform to external partner consumption formats. It operates at the intersection of spatial data, pipeline orchestration, and cloud delivery. diff --git a/docs/ADRs/002_topology_and_dependency_rules.md b/docs/ADRs/002_topology_and_dependency_rules.md index 33a7508..a9f2b4f 100644 --- a/docs/ADRs/002_topology_and_dependency_rules.md +++ b/docs/ADRs/002_topology_and_dependency_rules.md @@ -1,11 +1,20 @@ # ADR-002: Topology and Dependency Rules -**Status:** Accepted +**Status:** Accepted — amended 2026-06-27 (see [Amendment](#amendment-2026-06-27)) **Date:** 2026-06-02 **Deciders:** Project maintainers (PRIO MD&D Team) --- +> **Note (2026-06-27):** the *decision* below — dependencies must be acyclic and flow +> downward — still holds. Its **illustrative internal layering** (Pipeline Managers → +> Spatial Mapping Engine → Geographic Data Assets) is stale: the Spatial Mapping Engine and +> Geographic Data Assets layers were removed (ADR-011 / C-39). The current internal structure +> and the cross-repo topology this ADR originally omitted are in the +> [Amendment](#amendment-2026-06-27) at the bottom. Original text preserved as the record. + +--- + ## Context In complex systems, architectural fragility often emerges not from incorrect @@ -115,3 +124,56 @@ It does not define: Topology governs structure. Contracts govern interaction. + +--- + +## Amendment (2026-06-27) + +This amendment updates the *illustration* of the dependency rule to current reality and adds +the **cross-repo topology** the original ADR omitted. The rule itself (acyclic, downward) is +unchanged. + +### Cross-repo topology + +views-postprocessing is one stage in a one-way platform pipeline. Dependencies flow **down**: + +``` +views-datafactory (produces data) + ↓ +views-pipeline-core (the framework: lifecycle, data loader, dataset, datastore tools) + ↓ +views-postprocessing (THIS REPO — post-forecast delivery + input-integrity) + ↓ +views-faoapi (serves the delivered data; collapses draws) +``` + +- This repo **depends on** views-pipeline-core (it subclasses its postprocessor base — + Template Method) and views-frames (the frame contract). It **does not** depend on faoapi. +- **views-pipeline-core does not depend on this repo** (verified: zero imports). The + dependency is strictly one-way; a cycle here would be an architectural defect per the rule + above. +- **views-models** is the runner/composition root — it constructs this repo's manager and + calls `.execute()`; it is not a dependency *of* this repo. + +### Current internal layering (replaces the stale illustration) + +``` +Pipeline Manager (UNFAOPostProcessorManager — orchestration; a pipeline-core subclass) + ↓ calls +Delivery Invariants (views_postprocessing/delivery/ — representation-free rules) + ↑ fed primitives by +Representation Seam (unfao/extraction.py — the only pandas-aware module) + ↓ alongside +Enrichment (GaulLookupEnricher + data/gaul_lookup.parquet) · Producer Data-Facts (source_metadata.py) +``` + +- The manager **calls** the delivery invariants; it never inherits them. Invariants depend + only on primitives (DIP), so a representation change lands in the seam alone (OCP / C-40). +- Geography is a precomputed asset consumed by the enricher — there is no runtime spatial + engine layer anymore. + +### See also + +- The narrative version of this topology and the internal seams: + [`docs/architecture/role_and_seams.md`](../architecture/role_and_seams.md). +- The revised ontology these categories come from: [ADR-012](012_revised_ontology.md). diff --git a/docs/ADRs/012_revised_ontology.md b/docs/ADRs/012_revised_ontology.md new file mode 100644 index 0000000..3c66894 --- /dev/null +++ b/docs/ADRs/012_revised_ontology.md @@ -0,0 +1,83 @@ +# ADR-012: Revised ontology of views-postprocessing (post-lookup migration) + +**Status:** Accepted +**Date:** 2026-06-27 +**Deciders:** Project maintainers (PRIO MD&D Team) +**Supersedes:** [ADR-001](001_ontology_of_views_postprocessing.md) + +--- + +## Context + +[ADR-001](001_ontology_of_views_postprocessing.md) defined the repository's ontology around +a **runtime spatial-mapping engine** and **bundled shapefiles** as the authoritative, +stable core. That architecture no longer exists: + +- [ADR-011](011_replace_runtime_mapper_with_precomputed_lookup.md) replaced the runtime + geopandas mapper with a **precomputed GAUL lookup table** (`GaulLookupEnricher`). +- The mapper, shapefiles, and caching machinery were deleted (C-39 / PR #42). +- Input-integrity invariants and structured delivery were added (epic #51). + +So ADR-001's core categories ("Geographic Data Assets", "Spatial Mapping Engine") describe +deleted code. This ADR restates the ontology to match what the repository **actually is**: +a **post-forecast delivery + input-integrity layer**, not a spatial-mapping library and not +a statistical post-processor. (For the wider narrative — role vs the sibling repos and the +internal seams — see [`docs/architecture/role_and_seams.md`](../architecture/role_and_seams.md).) + +--- + +## Decision + +The repository defines a **closed set of conceptual categories**. Anything that does not +clearly belong to one is out of scope and must be redesigned or rejected. + +### Core ontological categories (current) + +| Category | Purpose | Authority | Stability | +|----------|---------|-----------|-----------| +| **Delivery Invariants** | Representation-free rules over primitives that a delivery must satisfy (coverage, forecast identity, observed-range, provenance). Live in `views_postprocessing/delivery/`. | Authoritative — they define what a valid delivery is | Stable — changes are governance decisions | +| **Representation Seam** | The single pandas-aware module (`unfao/extraction.py`) that turns the delivery's external representation into the primitives the invariants consume. | Derived — isolates the representation so invariants stay representation-free | Evolving — the one place a representation change (e.g. pandas → frames, C-40) lands | +| **Enrichment Asset + Engine** | The precomputed GAUL lookup (`data/gaul_lookup.parquet`) and the merge that joins it (`GaulLookupEnricher`). | Authoritative for geographic metadata | Stable — the lookup is rebuilt only when the producer (datafactory) releases new GAUL data | +| **Pipeline Manager** | The thin `UNFAOPostProcessorManager` — a concrete pipeline-core postprocessor (Template-Method subclass) that *orchestrates* read/transform/validate/save and **calls** the invariants (never inherits them). | Derived — implements delivery using the categories above | Evolving — changes as partner requirements change | +| **Producer Data-Facts** | Data-related facts sourced straight from the producer (datafactory) — e.g. `last_valid_month_id` — isolated in `unfao/source_metadata.py`. | Authoritative (the producer is the source of truth, D-07) | Evolving | +| **External Service Configurations** | Appwrite connection configs / env vars / bucket references. | Operational | Evolving | +| **Derived Outputs** | Enriched parquet files produced per run and delivered to the partner store. | Ephemeral | Ephemeral | + +### Explicitly *not* in this repository's ontology + +- **Spatial intersection / shapefiles / a runtime mapping engine** — removed (ADR-011/C-39); + geography is precomputed upstream and consumed as a lookup. +- **Statistical post-processing** — draw-collapse (MAP/HDI) is downstream in views-faoapi + (`views_frames_summarize`); reconciliation is in `views_frames_reconcile`. This repo + *preserves* forecast values and delivers them; it does not transform them. + +--- + +## Rationale + +- **Screaming architecture:** the categories now match the package layout — `delivery/` + (invariants), `unfao/extraction.py` (seam), `unfao/enrichment.py` + `data/` (enrichment), + `unfao/managers/` (orchestration). A reader can infer responsibilities from the structure. +- **DIP / OCP:** primitives are the abstraction the invariants depend on; the representation + seam is the single point of change for a representation migration (C-40), so the invariants + are closed against it. +- **Honest scope:** naming the repo a delivery + integrity layer (not a postprocessor that + does statistics) prevents the recurring confusion about where collapse/reconciliation live. + +--- + +## Consequences + +**Positive:** the ontology is verifiable against the code; onboarding docs (README, the +role-&-seams doc) can derive from it; the C-40 representation migration has a named seam. + +**Negative:** when the representation migrates (pandas → frames) this ADR's "Representation +Seam" wording will need a light touch (the seam stays; its internals change). + +--- + +## Notes + +- Stability is a design constraint, not a preference (inherited from ADR-001 §Stability Rules). +- Dependency direction and cross-repo topology are governed by [ADR-002](002_topology_and_dependency_rules.md). +- Class-level contracts are governed by ADR-006 (see `docs/CICs/`). diff --git a/docs/ADRs/README.md b/docs/ADRs/README.md index 9742cdc..01b192c 100644 --- a/docs/ADRs/README.md +++ b/docs/ADRs/README.md @@ -22,10 +22,12 @@ These ADRs define system philosophy and governance: Establishes the ADR practice itself. - **ADR-001** — Ontology of views-postprocessing - Defines what concepts exist. + ⚠️ **Superseded by ADR-012** (the original shapefile/spatial-engine ontology; kept as the + historical record). - **ADR-002** — Topology and Dependency Rules - Defines structural dependency direction. + Defines structural dependency direction. **Amended 2026-06-27** with the cross-repo + topology + current internal layering. - **ADR-003** — Authority of Declarations Over Inference Defines where semantic authority lives. @@ -59,6 +61,11 @@ These ADRs form the architectural constitution of the repository. - **ADR-011** — Replace Runtime Mapper with Precomputed Lookup Table Replaces the 3,122-line runtime spatial mapper with a ~65K-row precomputed Parquet lookup table. Area-majority assignment confirmed as FAO contractual requirement. +- **ADR-012** — Revised Ontology (post-lookup migration) + Supersedes ADR-001. Restates the ontology to the current reality: a post-forecast delivery + + input-integrity layer (delivery invariants, the representation seam, the lookup + enrichment, the thin manager) — no runtime spatial engine. + ADRs numbered 010 and above define: - Domain-specific decisions