Context
While scoping #3485 (TRACE TRO emission for webapp simulation runs) I found that policyengine-api is pinned to policyengine>0.12.0,<1 and imports modules from the pre-v4 orchestrator that do not exist in current pe.py (4.3.1+):
policyengine.simulation.SimulationOptions (used in policyengine_api/services/economy_service.py:23)policyengine.utils.data.datasets.get_default_dataset (used at policyengine_api/services/economy_service.py:24)
Both were removed during the 0.x → 4.x rearchitecture. The new v4 provenance primitives we need for TRACE TRO emission (policyengine.provenance.trace.build_trace_tro_from_release_bundle, build_simulation_trace_tro) only exist on the v4 line.
What needs to happen
Migrate the api from policyengine 0.x → 4.3.1+. At a minimum:
- Replace
SimulationOptions usages with the v4 equivalent (whatever the v4 unified orchestrator exposes for simulation configuration). See src/policyengine/simulation.py in the policyengine.py repo for the current shape.
- Replace
get_default_dataset usages with the v4 dataset resolution path. See src/policyengine/tax_benefit_models/common/model_version.py and src/policyengine/provenance/manifest.py for how v4 resolves certified data.
- Bump
policyengine-us → 1.653.3+ and policyengine-uk → 2.88.0+ (the versions v4 is certified against).
- Update any other internal APIs that changed between v0.x and v4.
- Re-run the full integration test suite; simulation output numbers should not change if the migration is done correctly (v4 is a thin orchestrator over the same country packages; model math is unchanged).
Why this is a prerequisite
TRACE TRO emission requires calling pe.py v4's provenance helpers, and vendoring those into the api would fork the canonical JSON bytes (hashes would not match). The migration therefore needs to land before #3485 is implementable.
Scope
This issue is scoped to just the version bump and the pre-v4 → v4 API mapping. Not scoped: any redesign of how the api sets up simulations, any new features, any refactor beyond what's needed to get the test suite green on v4.
Related
Context
While scoping #3485 (TRACE TRO emission for webapp simulation runs) I found that policyengine-api is pinned to
policyengine>0.12.0,<1and imports modules from the pre-v4 orchestrator that do not exist in current pe.py (4.3.1+):policyengine.simulation.SimulationOptions(used inpolicyengine_api/services/economy_service.py:23)policyengine.utils.data.datasets.get_default_dataset(used atpolicyengine_api/services/economy_service.py:24)Both were removed during the 0.x → 4.x rearchitecture. The new v4 provenance primitives we need for TRACE TRO emission (
policyengine.provenance.trace.build_trace_tro_from_release_bundle,build_simulation_trace_tro) only exist on the v4 line.What needs to happen
Migrate the api from
policyengine0.x → 4.3.1+. At a minimum:SimulationOptionsusages with the v4 equivalent (whatever the v4 unified orchestrator exposes for simulation configuration). Seesrc/policyengine/simulation.pyin the policyengine.py repo for the current shape.get_default_datasetusages with the v4 dataset resolution path. Seesrc/policyengine/tax_benefit_models/common/model_version.pyandsrc/policyengine/provenance/manifest.pyfor how v4 resolves certified data.policyengine-us→ 1.653.3+ andpolicyengine-uk→ 2.88.0+ (the versions v4 is certified against).Why this is a prerequisite
TRACE TRO emission requires calling pe.py v4's provenance helpers, and vendoring those into the api would fork the canonical JSON bytes (hashes would not match). The migration therefore needs to land before #3485 is implementable.
Scope
This issue is scoped to just the version bump and the pre-v4 → v4 API mapping. Not scoped: any redesign of how the api sets up simulations, any new features, any refactor beyond what's needed to get the test suite green on v4.
Related
policyengine.pyrelease notes for 4.0–4.3.1 (breaking changes)