Skip to content

S6 — forecast outbound → arrow sample-frame wire (ships (N,S) to FAO) #91

Description

@Polichinel

Epic: #85 · S6 · [CROSS-REPO-GATED] · blocked · depends on #88, faoapi #45, register C-40

Background

This is the step that actually propagates (N,S) to FAO. Today _save writes object-dtype df.to_parquet (unfao.py:298,309) to Appwrite, and views-faoapi reads it (pd.read_parquetto_array_columns decodes object-dtype cells → numpy → build_prediction_frame). The forecast parquet is a wire contract vpp cannot change unilaterally.

Work (gated — coordinated two-sided cutover)

  • vpp _save: emit the forecast as a views-frames arrow sample-frame (views_frames.io.arrow.save — flat columnar time/unit/sample/value) instead of object-dtype parquet.
  • views-faoapi: ingest via arrow.loadPredictionFrame instead of pd.read_parquetto_array_columnsbuild_prediction_frame.

Acceptance criteria

  • Forecast delivery carries (N,S) uncollapsed as an arrow sample-frame; faoapi reconstructs the PredictionFrame.
  • On S=1 inputs the reconstructed output is identical to today's point delivery (regression).
  • Cutover is coordinated (faoapi can read the new format before vpp stops writing the old).

Parity / validation

Dual-write / golden-file during cutover: emit both legacy object-dtype parquet and the arrow sample-frame; assert faoapi's reconstructed PredictionFrame is array-equal from both. S1's conformance harness is the shared contract both repos test against.

Dependencies / gate

Blocked on: S3 (interior already a PredictionFrame), faoapi #45 (consumer half + ADR-046 amendment), and register C-40. Never change the emitted wire before faoapi can read it.

Files

views_postprocessing/unfao/managers/unfao.py (_save); coordinated changes in views-faoapi (tracked there). Target API: views_frames.io.arrow.


Review amendment (expert-code-review, 2026-06-28) — wire I/O behind a sink adapter

Emit the arrow sample-frame through a small delivery sink adapter the manager _save calls, not inline to_parquet/arrow.save in the method.

  • Why: same SRP/DIP reason as S3 (S3 — forecast convert-at-the-door (internal PredictionFrame) #88) — keep representation/I/O out of the manager god-method (register C-40 thin-shell). The sink adapter is the symmetric counterpart to S3's source adapter.
  • Acceptance criteria addition:
    • Outbound arrow serialization lives behind a delivery sink adapter; _save calls it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedBlocked on a dependency or decisionimplementationCode implementation workneeds-decisionRequires a human decision before proceedingstoryA single reviewable unit of an epic

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions