Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# ADR-011: Replace Runtime Spatial Mapper with Precomputed Lookup Table

<!-- legacy-ok-file: this ADR IS the decision to retire the runtime mapper. `PriogridCountryMapper`, `mapping.py`, `geopandas` and the cache machinery are its subject, named throughout and correctly. Marking the document once says that where a reader sees it; marking fourteen lines would be noise pretending to be rigour. -->

**Status:** Accepted
**Date:** 2026-06-02
**Deciders:** Project maintainers (PRIO MD&D Team)
Expand Down
10 changes: 5 additions & 5 deletions docs/ADRs/012_revised_ontology.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ a **runtime spatial-mapping engine** and **bundled shapefiles** as the authorita
stable core. That architecture no longer exists:

- [ADR-011](011_replace_runtime_mapper_with_precomputed_lookup.md) replaced the runtime
geopandas mapper with a **precomputed GAUL lookup table**.
geopandas mapper with a **precomputed GAUL lookup table**. <!-- legacy-ok: records what ADR-011 replaced; the mapper is the predecessor, not the present -->
- The mapper, shapefiles, and caching machinery were deleted (C-39 / PR #42).
- Input-integrity invariants and structured delivery were added (epic #51).

Expand Down Expand Up @@ -58,9 +58,9 @@ described the cut-a-repo case; it now says which is which.
| Category | Purpose | Authority | Stability |
|----------|---------|-----------|-----------|
| **Delivery Invariants** | Representation-free rules over primitives that a delivery must satisfy: coverage, no-collapse, gid parity, observed-range, provenance. Live in `delivery/` — **nothing there imports pandas or views_frames**. *Forecast identity was one of these until 2026-07-31 — see the amendment below.* | Authoritative — they define what a valid delivery is | Stable — changes are governance decisions |
| **Representation Seam** | `contract/frame_extraction.py` — turns a `views_frames` frame into the primitives the invariants consume. **One seam.** Its pandas sibling `unfao/extraction.py` was deleted in #151 once the pandas delivery was retired; the two ran as deliberate WET siblings through the migration. | Derived — isolates the representation so invariants stay representation-free | Evolving |
| **Representation Seam** | `contract/frame_extraction.py` — turns a `views_frames` frame into the primitives the invariants consume. **One seam.** Its pandas sibling `unfao/extraction.py` was deleted in #151 once the pandas delivery was retired; the two ran as deliberate WET siblings through the migration. | Derived — isolates the representation so invariants stay representation-free | Evolving | <!-- legacy-ok: states that the pandas sibling WAS deleted in #151 — the retirement is the claim -->
| **Wire Mechanism** | `contract/wire/` — the ADR-013 contract: header, shard, sidecar, run manifest, sink, source selection. Partner-neutral: it takes its consumer name and collapse floor as **arguments** (#153). | Authoritative — the contract with the consumer | Stable — changes are contract amendments |
| **Enrichment Asset** | The precomputed GAUL lookup (`data/gaul_lookup.parquet`), its identity in `contract/gaul_lookup.py`, its schema in `contract/gaul_schema.py`, and the keyed gathers that join it — `contract/historical.py` for the actuals artifact and `contract/wire/sidecar.py` for the §5 sidecar. *(A second gather lived in `contract/enrichment.py` with no production caller; retired in #90, register **C-75**.)* | Authoritative for geographic metadata | Stable — rebuilt only when the producer releases new GAUL data |
| **Enrichment Asset** | The precomputed GAUL lookup (`data/gaul_lookup.parquet`), its identity in `contract/gaul_lookup.py`, its schema in `contract/gaul_schema.py`, and the keyed gathers that join it — `contract/historical.py` for the actuals artifact and `contract/wire/sidecar.py` for the §5 sidecar. *(A second gather lived in `contract/enrichment.py` with no production caller; retired in #90, register **C-75**.)* | Authoritative for geographic metadata | Stable — rebuilt only when the producer releases new GAUL data | <!-- legacy-ok: records the second gather that was retired in #90 -->
| **Artifact Builders** | `contract/historical.py` — turns a frame plus the lookup into the partner-facing artifact. | Derived | Evolving |
| **External Facts** | Facts read from systems this repo does not own: the producer's (`contract/source_metadata.py` — `last_valid_month_id`, D-07) and the store's (`contract/store_metadata.py`). | Authoritative (the owning system is the source of truth) | Evolving |
| **Launch Declarations** | `contract/launch_config.py` — the delivery mode the launcher must declare. Omitting a key is **refused by name**, never inferred (ADR-003, register C-63). | Authoritative | Stable |
Expand All @@ -70,7 +70,7 @@ described the cut-a-repo case; it now says which is which.

**Two claims this ADR made until 2026-08-01, both now corrected rather than quietly dropped**
(register C-67). It called the manager *"the **thin** `UNFAOPostProcessorManager`"* when it was
636 lines holding two of everything, and it called `unfao/extraction.py` *"the **single**
636 lines holding two of everything, and it called `unfao/extraction.py` *"the **single** <!-- legacy-ok: quotes the false claim this ADR was written to correct -->
pandas-aware module"* when pandas lived in three. Both drifted the same way: the ADR described
the intended end state of a migration that then stopped one step short. Both are now true —
pandas is **absent from the package entirely** (#89 reduced the last one to a type-only import; #90 retired the module that held it — register C-75) — and the load-bearing ones are
Expand Down Expand Up @@ -129,7 +129,7 @@ Seam" wording will need a light touch (the seam stays; its internals change).

## Amendment 2026-07-31 — forecast identity re-homed to the wire layer (#150, epic #148)

**`views_postprocessing/delivery/identity.py` is retired.** The "Delivery Invariants" row
**`views_postprocessing/delivery/identity.py` is retired.** The "Delivery Invariants" row <!-- legacy-ok: the amendment recording that identity.py is retired -->
above listed *forecast identity* among the authoritative rules; that module no longer
exists, and this amendment records where the rule went so a reader of the row is not
looking for deleted code.
Expand Down
10 changes: 5 additions & 5 deletions docs/ADRs/013_sampled_forecast_wire_contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -1120,7 +1120,7 @@ record execution progress against it.
**4.73 GB peak, 108 shards, 5 min, zero store calls** (interlock held).
**(B) the historical path is pandas-free** (#126): actuals fetched as a
`views_frames.FeatureFrame` (the frame path's FIRST production consumer —
C-40's gate lifted), artifact built by `unfao/historical.py` via pyarrow —
C-40's gate lifted), artifact built by `contract/historical.py` via pyarrow —
reader-level parity with a legacy characterization golden proven through
faoapi's own reader semantics. Two ghosts found in the legacy artifact and
deliberately exorcised (faoapi reader verified safe on both): junk `row`/`col`
Expand All @@ -1142,8 +1142,8 @@ record execution progress against it.
passed at 128 draws, sidecar + manifest staged, zero store calls.
- **2026-07-20 — HOP-B SINK LEG SHIPPED (epic #105 complete; upload-disabled).**
The contract's missing middle exists in fixture-proven code:
`unfao/wire/` (naming, header, shard, sidecar, run_manifest, source_selection,
sink) + `unfao/product.py` + `delivery/parity.py`, wired into the manager
`contract/wire/` (naming, header, shard, sidecar, run_manifest, source_selection,
sink) + each partner's `product.py` + `delivery/parity.py`, wired into the manager
behind an explicit declared `wire_contract` launch key. Settled by shipping:
**§4.2a's configuration home is `unfao/product.py`**; **§5.2's parity
invariant is `delivery/parity.py`**; the §11.4 upload interlock is live in
Expand Down Expand Up @@ -1177,7 +1177,7 @@ record execution progress against it.
merged 2026-07-15 and run-0 uploaded 2026-07-27, so the sequencing constraint was
satisfied. The legacy reader it protected has now been **deleted**: this repo
reads forecasts only through the contract path (manifest selection,
`unfao/wire/source_selection.py`) and historical actuals only as a
`contract/wire/source_selection.py`) and historical actuals only as a
`views_frames.FeatureFrame` (#126).

**The retired guard, recorded here because the code that carried it is gone.**
Expand All @@ -1204,7 +1204,7 @@ record execution progress against it.
this ADR should know the surrounding code changed shape and the wire did not.

**What changed.** The pandas delivery this contract replaced was retired (#149) along
with the config fork that silently selected it; `delivery/identity.py` was retired
with the config fork that silently selected it; `delivery/identity.py` was retired <!-- legacy-ok: post-adoption record of what was retired and when -->
because the contract path enforces declared identity **per shard header** instead
(#150, §4.2a); the duplicate representation seam collapsed (#151); the GAUL lookup
became one artifact read once (#152); and the partner-neutral machinery — the whole
Expand Down
4 changes: 2 additions & 2 deletions docs/CICs/UNFAOPostProcessorManager.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
---

> **Corrected 2026-08-03 — this document named a collaborator the manager has never
> called.** Six statements described enrichment as delegated to `GaulLookupEnricher`,
> one naming the call `GaulLookupEnricher.enrich_dataframe_with_pg_info()`. The manager
> called.** Six statements described enrichment as delegated to `GaulLookupEnricher`, <!-- legacy-ok: the 2026-08-03 correction note; naming the error is the record -->
> one naming the call `GaulLookupEnricher.enrich_dataframe_with_pg_info()`. The manager <!-- legacy-ok: the 2026-08-03 correction note; naming the error is the record -->
> contains **zero** references to it — `tests/test_gaul_lookup_access.py` actively
> asserts its absence — and the sibling CIC has long said *"the manager does not call
> this class."* Two contract documents asserted opposite things about the same call.
Expand Down
Loading
Loading