Skip to content

feat: geofence-based geographic targeting for programs#76

Draft
jeremi wants to merge 105 commits into
19.0from
feature/program-geofence
Draft

feat: geofence-based geographic targeting for programs#76
jeremi wants to merge 105 commits into
19.0from
feature/program-geofence

Conversation

@jeremi
Copy link
Copy Markdown
Member

@jeremi jeremi commented Mar 6, 2026

Summary

  • Add spp_program_geofence module for geofence-based program targeting and eligibility management.
  • Extend spp_gis spatial operators to support MultiPolygon and GeometryCollection.
  • Make MapTiler API key optional in GIS UI, with OSM fallback when key is not configured.
  • Stabilize GIS edit/map widgets (re-render lifecycle, draw interaction behavior).

Follow-up Fixes From Review

  • Fixed complex-geometry distance behavior in GIS domain operators:
    • For MultiPolygon / GeometryCollection, (geojson, distance) now applies ST_Buffer(...) correctly instead of ignoring the distance operand.
    • Keeps existing SRID handling parity by transforming to EPSG:3857 when SRID is 4326.
  • Added regression tests for distance-buffer behavior on both complex geometry types.
  • Aligned access-rights and menu visibility:
    • Added explicit read ACLs in spp_program_geofence for spp.gis.geofence and spp.gis.geofence.tag for Programs Viewer/Validator roles.
    • Added groups restriction on the Geofences menu to avoid exposing menu items to users lacking read access.

Notable Functional Additions

spp_program_geofence

  • Program-level geofence_ids and geofence_count.
  • Geofence eligibility manager (spp.program.membership.manager.geofence) with:
    • Tier 1: coordinate intersection.
    • Tier 2: area intersection fallback (optional, with area type filter).
  • Program configuration UI integration and program creation wizard support.
  • Geofence management views/action/menu.

spp_gis

  • Operator support for complex geometry types via ST_GeomFromGeoJSON.
  • OSM style fallback in renderer/edit widgets when MapTiler key is missing.
  • Placeholder-key handling in controller (YOUR_MAPTILER_API_KEY_HERE treated as unconfigured).

Test Plan

  • Added unit tests for:
    • MultiPolygon/GeometryCollection SQL generation.
    • Complex geometry distance-buffer SQL path.
  • Local validation in this environment:
    • python3 -m py_compile spp_gis/operators.py spp_gis/tests/test_geo_fields.py
  • Full module test execution via ./spp t ... is environment-blocked here (tomllib missing in local Python, and fallback script depends on shuf).

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a robust geofence-based geographic targeting system for programs, allowing for precise and flexible definition of eligibility zones. It significantly enhances the underlying GIS infrastructure by adding support for complex geometric types and improving the reliability and user experience of map widgets. These changes collectively empower programs with more advanced spatial management tools and ensure a more secure and resilient mapping environment.

Highlights

  • New Geofence-Based Targeting Module: Introduced a new spp_program_geofence module that enables programs to define geographic boundaries (geofences) for registrant eligibility.
  • Hybrid Two-Tier Spatial Targeting: Implemented a sophisticated eligibility manager that uses a two-tier approach: precise GPS coordinates and an administrative area fallback for registrants without GPS data.
  • Enhanced GIS Capabilities: Upgraded spp_gis to support MultiPolygon and GeometryCollection GeoJSON types, significantly expanding the flexibility of spatial queries.
  • Map Widget Improvements and Fixes: Made MapTiler API key optional with an OpenStreetMap (OSM) raster tile fallback, and resolved several map widget bugs including WebGL context leaks, draw control stacking, and layer removal issues.
  • SQL Injection Prevention: Addressed potential SQL injection vulnerabilities in GeoJSON operators by implementing parameterized queries for geometry creation.
Changelog
  • spp_gis/operators.py
    • Added support for 'MultiPolygon' and 'GeometryCollection' types in ALLOWED_LAYER_TYPE.
    • Introduced create_from_geojson method to safely construct geometry from GeoJSON using parameterized SQL queries.
    • Updated validate_geojson to accept 'MultiPolygon' and 'GeometryCollection' types.
    • Modified domain_query to utilize create_from_geojson for complex geometry types.
  • spp_gis/static/src/js/views/gis/gis_renderer/gis_renderer.esm.js
    • Made MapTiler API key configuration conditional, allowing map rendering without it.
    • Implemented OpenStreetMap (OSM) raster tile fallback when a MapTiler API key is not provided.
    • Made GeocodingControl conditional on the presence of a MapTiler API key.
    • Changed API key fetching error logging from error to warn.
  • spp_gis/static/src/js/widgets/gis_edit_map/field_gis_edit_map.esm.js
    • Made MapTiler API key configuration conditional.
    • Introduced _getMapStyle method to provide OSM raster tile fallback when a MapTiler API key is not available.
    • Added this.map.remove() before creating a new map instance to prevent WebGL context leaks.
    • Updated removeSourceAndLayer to correctly remove specific polygon, point, and linestring layers.
    • Added this.map.removeControl(this.draw) to prevent draw control stacking.
    • Removed debug console.log statements and a placeholder image popup on map click.
    • Changed API key fetching error logging from error to warn.
  • spp_gis/tests/test_geo_fields.py
    • Added TestOperatorMultiPolygon class to test MultiPolygon and GeometryCollection types.
    • Verified domain_query correctly handles MultiPolygon and GeometryCollection GeoJSON, including parameterized SQL and table-qualified column names.
    • Confirmed domain_query accepts GeoJSON strings and Shapely objects for complex types.
    • Ensured validate_geojson accepts new geometry types and rejects invalid ones.
    • Verified that Polygon queries continue to use coordinate-based construction.
  • spp_program_geofence/DESCRIPTION.md
    • Added a detailed description of the new module's features and limitations.
  • spp_program_geofence/init.py
    • Initialized the Python package for the new module.
  • spp_program_geofence/manifest.py
    • Defined the module's metadata, dependencies, and data files.
  • spp_program_geofence/models/init.py
    • Imported eligibility_manager and program models.
  • spp_program_geofence/models/eligibility_manager.py
    • Registered 'Geofence Eligibility' as a new manager type.
    • Implemented GeofenceMembershipManager with include_area_fallback and fallback_area_type_id fields.
    • Added _get_combined_geometry to union geofence geometries using shapely.ops.unary_union.
    • Implemented _find_eligible_registrants for two-tier spatial eligibility (GPS coordinates and administrative area fallback).
    • Provided methods for enroll_eligible_registrants, verify_cycle_eligibility, and import_eligible_registrants (including asynchronous import for large datasets).
    • Added action_preview_eligible for UI preview of eligible registrants.
  • spp_program_geofence/models/program.py
    • Extended spp.program model with geofence_ids (Many2many) and geofence_count fields.
    • Added _compute_geofence_count and action_open_geofences methods.
  • spp_program_geofence/security/ir.model.access.csv
    • Defined access control rules for the spp.program.membership.manager.geofence model for different user roles.
  • spp_program_geofence/static/description/index.html
    • Added a brief HTML description for the module.
  • spp_program_geofence/tests/init.py
    • Initialized the test suite for the module.
  • spp_program_geofence/tests/test_geofence_eligibility.py
    • Added TestGeofenceEligibility class with comprehensive tests for the geofence eligibility manager's logic.
    • Tested Tier 1 (coordinates) and Tier 2 (area fallback) eligibility, including various scenarios like multiple geofences, no geofences, disabled registrants, and target types.
    • Included tests for MultiPolygon geofences and the geofence_count field.
    • Added TestGeofenceEligibilityOfficer to verify access rights for program officers.
  • spp_program_geofence/views/eligibility_manager_view.xml
    • Defined the form view for the geofence eligibility manager, including settings for area fallback, program geofence display, and a preview section.
  • spp_program_geofence/views/geofence_view.xml
    • Defined list, form, and search views for spp.gis.geofence and added a menu item under 'Area'.
  • spp_program_geofence/views/program_view.xml
    • Extended the program form view to include a stat button for geofence count and a section on the 'Overview' tab for managing geofence_ids.
Activity
  • All 86 tests for spp_gis passed successfully.
  • All 23 tests for spp_program_geofence passed successfully.
  • Code quality checks (linters: ruff, ruff-format, prettier) passed.
  • Manual UI testing was performed to verify geofence creation, configuration of the geofence eligibility manager, previewing, and importing registrants.
  • Map display functionality was verified with and without the MapTiler API key.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a major new feature for geofence-based geographic targeting, including a new spp_program_geofence module and extensive tests. It also enhances the spp_gis module by adding support for MultiPolygon and GeometryCollection, and crucially fixes a potential SQL injection vulnerability by using parameterized queries. The map widgets are improved by making the MapTiler API key optional with an OpenStreetMap fallback, and several pre-existing bugs related to WebGL context leaks and control stacking are fixed. The changes are well-structured and of high quality. I have a few suggestions to improve maintainability and robustness.

Comment on lines 249 to 254
function updateArea(e) {
console.log(e);
var data = self.draw.getAll();
self.props.record.update({
[self.props.name]: JSON.stringify(data.features[0].geometry),
});
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The updateArea function assumes that self.draw.getAll().features will always contain at least one feature. If it's empty for any reason, data.features[0] will be undefined and accessing .geometry will cause a runtime error. It's safer to add a check.

Additionally, the draw.create and draw.update events pass the affected features in the event object e, which is more direct and efficient to use than calling getAll().

        function updateArea(e) {
            const features = e.features;
            if (features && features.length > 0) {
                self.props.record.update({
                    [self.props.name]: JSON.stringify(features[0].geometry),
                });
            }
        }

Comment on lines +190 to +193
if ben_count < 1000:
self._import_registrants(new_beneficiaries, state=state, do_count=True)
else:
self._import_registrants_async(new_beneficiaries, state=state)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The threshold 1000 for switching to asynchronous import, and the chunk size 10000 used in _import_registrants_async (line 203), are magic numbers. It would be better to define them as constants at the class or module level, for example ASYNC_IMPORT_THRESHOLD = 1000 and IMPORT_CHUNK_SIZE = 10000. This improves readability and makes these values easier to find and adjust.

Comment on lines +89 to +115
_getMapStyle() {
if (this.mapTilerKey) {
return maptilersdk.MapStyle.STREETS;
}
// Fallback: OSM raster tiles (no API key required)
return {
version: 8,
sources: {
osm: {
type: "raster",
tiles: ["https://tile.openstreetmap.org/{z}/{x}/{y}.png"],
tileSize: 256,
attribution:
'&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors',
},
},
layers: [
{
id: "osm-tiles",
type: "raster",
source: "osm",
minzoom: 0,
maxzoom: 19,
},
],
};
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The code to generate the fallback OpenStreetMap map style object is duplicated from spp_gis/static/src/js/views/gis/gis_renderer/gis_renderer.esm.js. To improve maintainability and avoid inconsistencies, this object could be defined as a constant or returned by a utility function in a shared file, and then imported in both field_gis_edit_map.esm.js and gis_renderer.esm.js.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 6, 2026

Codecov Report

❌ Patch coverage is 71.76518% with 707 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.13%. Comparing base (e9dad7e) to head (44102c0).

Files with missing lines Patch % Lines
spp_mis_demo_v2/models/mis_demo_generator.py 35.47% 231 Missing ⚠️
spp_gis_report/models/gis_report.py 53.43% 95 Missing ⚠️
spp_api_v2_gis/routers/ogc_features.py 26.13% 65 Missing ⚠️
spp_cel_domain/models/cel_translator.py 65.46% 48 Missing ⚠️
spp_api_v2_gis/services/ogc_service.py 85.81% 40 Missing ⚠️
spp_program_geofence/models/eligibility_manager.py 69.69% 40 Missing ⚠️
spp_demo_phl_luzon/models/area_loader.py 58.06% 39 Missing ⚠️
spp_api_v2_gis/routers/processes.py 77.27% 25 Missing ⚠️
spp_api_v2_gis/services/layers_service.py 0.00% 23 Missing ⚠️
spp_metric_service/models/demographic_dimension.py 84.87% 18 Missing ⚠️
... and 14 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             19.0      #76      +/-   ##
==========================================
+ Coverage   73.06%   73.13%   +0.06%     
==========================================
  Files        1069     1088      +19     
  Lines       62120    64398    +2278     
==========================================
+ Hits        45390    47099    +1709     
- Misses      16730    17299     +569     
Flag Coverage Δ
spp_analytics 93.13% <ø> (ø)
spp_api_v2 80.44% <91.17%> (+0.11%) ⬆️
spp_api_v2_change_request 66.85% <ø> (ø)
spp_api_v2_cycles 71.12% <ø> (ø)
spp_api_v2_data 64.41% <ø> (ø)
spp_api_v2_entitlements 70.19% <ø> (ø)
spp_api_v2_gis 79.02% <81.74%> (+7.49%) ⬆️
spp_base_common 90.26% <ø> (ø)
spp_cel_domain 61.35% <67.56%> (+0.18%) ⬆️
spp_demo_phl_luzon 69.85% <69.85%> (?)
spp_gis 75.30% <92.00%> (+1.20%) ⬆️
spp_gis_report 77.60% <53.19%> (-5.01%) ⬇️
spp_hazard 97.60% <94.02%> (-1.99%) ⬇️
spp_hazard_programs 97.14% <ø> (ø)
spp_metric_service 88.91% <86.86%> (-0.67%) ⬇️
spp_mis_demo_v2 68.11% <38.86%> (-5.50%) ⬇️
spp_program_geofence 74.25% <74.25%> (?)
spp_programs 64.84% <ø> (ø)
spp_registry 86.83% <ø> (ø)
spp_security 66.66% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
spp_api_v2/middleware/auth.py 76.11% <100.00%> (+0.36%) ⬆️
spp_api_v2/models/fastapi_endpoint_registry.py 94.59% <100.00%> (+0.30%) ⬆️
spp_api_v2/schemas/__init__.py 100.00% <100.00%> (ø)
spp_api_v2/schemas/bundle.py 100.00% <100.00%> (ø)
spp_api_v2_gis/__manifest__.py 0.00% <ø> (ø)
spp_api_v2_gis/models/__init__.py 100.00% <100.00%> (ø)
spp_api_v2_gis/models/api_client_scope.py 100.00% <100.00%> (ø)
spp_api_v2_gis/models/fastapi_endpoint.py 100.00% <100.00%> (ø)
spp_api_v2_gis/routers/__init__.py 100.00% <100.00%> (ø)
spp_api_v2_gis/schemas/__init__.py 100.00% <100.00%> (ø)
... and 51 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jeremi
Copy link
Copy Markdown
Member Author

jeremi commented Mar 12, 2026

Addressed Gemini review feedback in commit 46c90b6:

  • Fixed draw handler robustness in GIS edit widget by using event features first and guarding geometry access (with fallback to draw state).
  • Replaced import magic numbers with named constants in geofence eligibility manager (ASYNC_IMPORT_THRESHOLD, IMPORT_CHUNK_SIZE).

Kept one medium suggestion as follow-up (optional): deduplicate identical OSM fallback style object between gis_renderer and field_gis_edit_map via shared utility. No functional risk from current duplication.

jeremi added 19 commits March 12, 2026 15:07
…operators

The Operator class only supported Point, LineString, and Polygon types
in domain queries. When shapely's unary_union creates a MultiPolygon
from non-overlapping polygons, the operator validation silently rejected
it, returning SQL("FALSE") and matching zero registrants.

Add ST_GeomFromGeoJSON path for complex geometry types that cannot be
easily constructed from coordinates.
New module that adds geofence-based geographic targeting to programs:
- Program-level geofence_ids field on Overview tab
- Geofence eligibility manager with hybrid two-tier spatial queries
  (GPS coordinates + administrative area fallback)
- Preview button showing matched registrant count
- Composable with other eligibility managers via AND logic
…ibility

- Use gis_intersects instead of gis_within for Tier 1 spatial query;
  gis_within generates ST_Within(value, field) which is backwards for
  point-in-polygon checks, while gis_intersects is symmetric
- Use disabled=None instead of disabled=False in domain (Datetime field)
- Use fields.Datetime.now() for disabled test data (not Boolean True)
- Use group_ids with Command.link() for Odoo 19 compatibility in tests
- Escape single quotes in create_from_geojson to prevent SQL injection
- Make preview_count/preview_error regular fields instead of computed;
  spatial queries now only run when the Preview button is clicked
- Use elif instead of two independent if statements for target_type
- Simplify _import_registrants loop to list comprehension
…ent UI

- Add fallback_area_type_id field to restrict Tier 2 area fallback to a
  specific administrative level (e.g. District), preventing overly broad
  matches from large provinces or regions
- Add geofence list/form/search views with menu under Area top-level,
  so users can browse and manage geofences independently
- Allow inline geofence creation from the program form
- Add 3 tests for area type filter behavior
When no MapTiler API key is configured, the map widget now falls back
to OpenStreetMap raster tiles instead of failing silently. This makes
the GIS features work out of the box without requiring a third-party
API key. Users who want vector tiles can still configure a MapTiler key.
…back

- Fix WebGL context leak: destroy previous map before creating new one
  in renderMap() to prevent accumulating WebGL contexts on onPatched
- Fix draw control stacking: remove previous MapboxDraw control before
  adding a new one in addDrawInteraction()
- Fix removeSourceAndLayer: remove all three layer IDs (polygon, point,
  linestring) instead of the source ID which doesn't match any layer
- Remove console.log debug statements from updateArea and onTrash
- Remove hardcoded laos_farm.png placeholder popup on polygon click
- Fix SQL injection in create_from_geojson: use SQL() with bound
  parameters instead of manual string escaping
- Apply OSM raster tile fallback to gis_renderer (matching edit widget)
- Guard GeocodingControl behind API key check in renderer
- Fix early-return-in-loop in eligibility manager methods with
  ensure_one()
- Log exceptions in preview instead of silently swallowing them
- Use efficient set lookup for beneficiary exclusion
- Use Command.set()/Command.clear() instead of tuple syntax in tests
The default system parameter value "YOUR_MAPTILER_API_KEY_HERE" was
being returned as a valid key, causing 403 errors from MapTiler instead
of falling back to OSM tiles.
renderMap() was overriding the OSM fallback style with a MapTiler
style reference when defaultRaster was set, even without an API key.
Guard the raster style override behind mapTilerKey check.
…nu order

The GeoPolygonField edit widget requires a GIS view (ir.ui.view with
type=gis) with data and raster layers to render the map. Without it,
opening a geofence form raised "No GIS view defined".

Also moved the Geofences menu item to sequence 200 so it appears last
in the Area menu.
When renderMap() destroys the old map, the MapboxDraw control's
internal map reference becomes null. Later, addDrawInteraction()
tried to removeControl(this.draw) from the new map, but the draw
control called this.map.off() on its now-null internal reference.

Fix: set this.draw = null before map.remove() so addDrawInteraction
skips the removeControl call for stale controls.
Move the Geographic Scope card from Overview to Configuration tab,
matching the card-based UI pattern. Add geofence_ids field to the
program creation wizard so geofences can be set during initial setup.
Existing geometry was added as a static map source/layer, making it
non-interactive: shapes couldn't be clicked, selected, or edited.
Now geometry is loaded into the MapboxDraw control via draw.add(),
enabling click-to-select, vertex editing, and trash deletion.
Also handles draw.delete event to clear the field value.
The tag_ids field on spp.gis.geofence was pointing to spp.vocabulary,
which is a generic vocabulary model containing all tag categories
(Country, Currency, etc.). Replace with a dedicated spp.gis.geofence.tag
model so the tags dropdown only shows geofence-specific tags.
- Remove dead methods from field_gis_edit_map (onLoadMap, addSourceAndLayer,
  addSource, addLayer, removeSourceAndLayer) no longer called after draw
  refactor
- Fix event listener stacking in addDrawInteraction: store handler refs
  and remove previous listeners before adding new ones, preventing
  duplicate record.update() calls when onUIChange() is called
- Fix disabled registrant filter: use ("disabled", "=", False) for
  consistency with DefaultEligibilityManager
@jeremi jeremi force-pushed the feature/program-geofence branch from 46c90b6 to 8e77302 Compare March 12, 2026 08:09
jeremi added 6 commits March 15, 2026 00:21
Add foundational components for OGC API - Processes (Part 1: Core):
- spp.gis.process.job model for async job tracking with job_worker
- Pydantic schemas for process list, description, execution, status
- ProcessRegistry service with dynamic indicator enum from spp.indicator
- Cron job for cleanup of stale/expired process jobs
- ACL entries for the new model
- processes.py: GET /gis/ogc/processes, GET /gis/ogc/processes/{id},
  POST /gis/ogc/processes/{id}/execution with sync/async support
- jobs.py: GET /gis/ogc/jobs, GET /gis/ogc/jobs/{id},
  GET /gis/ogc/jobs/{id}/results, DELETE /gis/ogc/jobs/{id}
- Wire routers into FastAPI endpoint registry
- Update OGC conformance to declare Processes classes
- Add processes link to OGC landing page
- Refactor statistics endpoint to delegate to ProcessRegistry
Tests cover:
- ProcessRegistry: list, describe, unknown ID, indicator enum, x-openspp-statistics
- Pydantic schemas: ProcessSummary, ExecuteRequest, StatusInfo, JobList
- spp.gis.process.job model: create, dismiss, stale cleanup, cron
- Input validation: single/batch geometry, bare arrays, limits, proximity
- HTTP integration: process list/describe, execution, async flow,
  job scoping, dismiss, conformance, landing page links
- Remove 'icon' field from test category creation (spp.metric.category
  does not have an icon field; Odoo 19 raises ValueError on unknown fields)
- Fix Odoo False-vs-None for empty Text fields: use `message or None`
  in _build_status_info to prevent Pydantic ValidationError when
  job.message is False instead of None
- Apply fix in both processes.py and jobs.py routers
- Restrict ACL: base.group_user gets read-only, base.group_system gets
  full CRUD (routers use sudo() for all mutations)
- Add UNIQUE constraint on job_id field
- Sanitize exception messages in job records to prevent leaking internals
- Extract shared helpers (_helpers.py): build_status_info, check_gis_scope,
  get_base_url used by both processes and jobs routers
- Extract process execution logic (process_execution.py): run_spatial_statistics,
  run_proximity_statistics used by both sync (router) and async (model) paths
- Add safe int parsing for ir.config_parameter values with fallback defaults
- Use SPATIAL_STATISTICS/PROXIMITY_STATISTICS constants instead of string literals
…OGC Processes

Address consumer feedback from the QGIS plugin team:

- Always include geometries_failed in batch summary responses (#1)
- Always populate computed_at with current UTC timestamp, even for empty results (#2)
- Add Retry-After: 5 header to async 201 responses and in-progress job status (#3/#8)
- Track batch progress per-geometry via on_progress callback in job execution (#4)
- Add x-openspp-batch-limit: 100 extension to spatial-statistics process description (#5)
jeremi and others added 28 commits April 2, 2026 16:18
…y scheme

The security scheme was declared as generic HTTPBearer, which didn't
tell API consumers that tokens come from the OAuth2 client credentials
flow at /oauth/token. Switched to FastAPI's OAuth2 class with
clientCredentials flow so the OpenAPI spec correctly advertises the
token endpoint and auth mechanism.
…OGC API

The geofence, proximity, and spatial_query routers were fully
duplicated by OGC API - Features and OGC API - Processes endpoints.
The QGIS plugin has already migrated to the OGC endpoints.

Removed: /gis/geofences, /gis/query/proximity, /gis/query/statistics
Kept: /gis/export/geopackage and /gis/statistics (unique, no OGC equivalent)
Schema fixes (geojson.py, ogc.py, processes.py):
- GeoJSON geometry type constrained to RFC 7946 Literal values
- Feature: add id, typed geometry, optional links
- FeatureCollection: replace non-standard metadata/styling with
  OGC timeStamp, numberMatched, numberReturned, links
- Add geofence-specific input models with extra="ignore" for QGIS
- CollectionInfo: add optional storageCrs
- ProcessSummary/ProcessList/ProcessDescription: add links
- ExecuteRequest.response: constrain to Literal["raw", "document"]

Router wiring (ogc_features.py, processes.py, jobs.py):
- POST/PUT geofences: switch from await request.json() to Pydantic
  Body() with CreateGeofenceInput/ReplaceGeofenceInput
- GET items/item: add responses with GeoJSON schema docs
- POST execution: add responses with oneOf result schemas
- GET job results: add responses with oneOf result schemas

Cleanup:
- Remove query.py from schemas/__init__ (only used by tests)
Add realistic examples via json_schema_extra to all key models:
- GeoJSONGeometry, GeoJSONFeature, GeoJSONFeatureCollection
- CreateGeofenceInput, ReplaceGeofenceInput
- ExecuteRequest (single, batch, and proximity variants)
- StatusInfo, SingleStatisticsResult, BatchStatisticsResult,
  ProximityResult

Examples are based on actual QGIS plugin usage patterns and
test fixtures to help API consumers understand request/response
shapes.
Add examples to LandingPage, Conformance, CollectionInfo (3 types:
report, geofences, data layer), Collections, ProcessSummary,
ProcessDescription, and ProcessList. Examples reflect actual service
output shapes and QGIS plugin usage patterns.
model_json_schema() produces $defs with local $ref paths that break
when inlined into OpenAPI responses. Switch to FastAPI's native
{"model": ...} in the responses dict, which properly registers
schemas in components/schemas with valid $ref pointers.
Replace hardcoded severity Selection with vocabulary-backed Many2one fields
using spp.vocabulary.code (CAP v1.2 standard). This enables future mapping
to other alerting protocols.

Changes:
- Add uuid field (auto-generated UUID4, indexed)
- Refactor severity to severity_id (Many2one to CAP severity vocabulary)
- Add cap_urgency_id, cap_certainty_id, cap_msg_type_id (vocabulary-backed)
- Add cap_event, effective, expires, source, source_alert_id fields
- Make category_id and start_date optional on model (required in form XML)
- Auto-populate start_date/end_date from effective/expires on create
- Create CAP vocabulary data (severity, urgency, certainty, msg_type)
- Add "Alert Details" tab to form view
- Update search view with new filter/grouping options
- Update demo data to use vocabulary code references
Implement reusable model methods for creating/updating incidents from
external alert systems (EWS), independent of the API layer:

- create_from_alert(): create incident + hazard_zone geofence + area linking
- update_from_alert(): update properties/geometry, handle cancellations
- _link_areas_from_geometry(): PostGIS ST_Intersects area auto-linking
- to_geojson(): GeoJSON Feature representation with CAP-aligned properties
- _get_alert_geometry(): resolve geometry from linked hazard_zone geofence

Includes comprehensive tests for all methods.
Implement OGC API Features Part 4 for hazard incidents, enabling EWS
systems to push alerts via standard GeoJSON endpoints.

API surface:
- POST /collections/incidents/items: create incident from alert
- PUT /collections/incidents/items/{uuid}: update incident
- GET /collections/incidents/items: list with filters
- GET /collections/incidents/items/{uuid}: single incident

Features:
- Duplicate detection: 409 Conflict with Location header on duplicate
  source_alert_id
- Named query filters: event, severity, status (incidents); incident_code
  (geofences); datetime (both)
- OGC datetime parameter parsing (instant, interval, open-ended)
- gis:incident scope for write access
- Pydantic schemas for input validation (CreateIncidentInput,
  ReplaceIncidentInput)
- Collection-specific POST/PUT routes for proper OpenAPI schema generation

Includes comprehensive service-level tests for all operations.
…ences

- Use datetime.fromisoformat() instead of fields.Datetime.from_string()
  to handle ISO 8601 format with T separator and Z timezone suffix
- Update spp_hazard_programs view to use severity_id (vocabulary-backed)
Critical:
- Generate fallback code when source_alert_id absent (prevents 500)
- Convert ISO 8601 datetimes to naive UTC for Odoo ORM compatibility
- Add missing base.group_user ACL for spp.hazard.impact
- Fix emergency ribbon placement (inside sheet, not before)
- Add .sudo() to VocabCode lookup in severity filter

Important:
- Narrow _link_areas_from_geometry exception to psycopg2.Error
- Replace deprecated name_get with _compute_display_name (Odoo 19)
- Merge cancel close into single ORM write (avoid double chatter)
- Add no_create/no_open options to all vocabulary Many2one fields
- Add group-by for cap_msg_type_id in search view
- Add active=True to bbox geofence search for consistency
- Add effective!=False guard in temporal filtering
- Add scope enforcement tests for incident endpoints
- Fix shared mutable state in TestUpdateFromAlert (setUp per test)
- Assert geometry actually changed in test_update_with_new_geometry

Minor:
- Move Command import to top-level
- Improve field labels (Effective From, Expires At)
- Add empty state message to Impacts tab
- Add optional attributes to list view columns
- Add typed input schemas for spatial/proximity statistics processes
- Type ExecuteRequest.inputs as union instead of bare dict
- Add OpenAPI examples to GeoJSON, OGC, and statistics schemas
- Fix relative imports in processes.py
- Add model_rebuild() calls for forward reference resolution
The typed union (SpatialStatisticsInputs | ProximityStatisticsInputs | dict)
caused Pydantic to parse inputs into model objects, breaking downstream code
that calls .get() on the inputs dict. The validation and execution layers
expect plain dicts throughout.
Configure Vite dev server proxy to route API requests to the openspp container.
…bodies

Provides a Pydantic Field factory plus a FastAPI OpenAPI hook that
documents a dict-typed request body as oneOf of typed models without
changing the runtime type. Models referenced via polymorphic_body are
registered in a module-level registry; the hook injects them into
components/schemas only for apps whose routes actually reference them,
avoiding cross-app pollution.

Unit tests cover runtime pass-through, oneOf/$ref rendering, per-app
scoping, nested model resolution, and optional default=None fields.
Wires install_polymorphic_openapi_hook into the api_v2 FastAPI app so
that any schema using polymorphic_body() gets its referenced models
injected into components/schemas. The hook is a no-op for consumers
that don't use polymorphic_body, so this change is safe to land before
any consumer adopts the helper.
Forward-looking guard for polymorphic_body usage: walks the live
OpenAPI schema and fails if any $ref under #/components/schemas/ points
to a schema that is not registered. Catches the class of regression
where someone forgets install_polymorphic_openapi_hook or renames a
referenced model.
…uest.inputs

Uses the new polymorphic_body helper so OpenAPI documents
SpatialStatisticsInputs and ProximityStatisticsInputs as oneOf
alternatives, while inputs stays a plain dict at runtime.

Reverts the documentation regression from 46c2c36 without
reintroducing the runtime breakage that prompted it.

Includes incidental ruff-format fixes to pre-existing multi-line
descriptions in the same file (flagged by pre-commit on touch).
…roup

Bundle service only accepts Individual and Group resources (see
spp_api_v2/services/bundle_service.py:299, 324, 350). Using
polymorphic_body makes OpenAPI document that contract explicitly while
resource stays dict[str, Any] | None at runtime, so validators and
placeholder-UUID resolution logic are unchanged.

Also reorders spp_api_v2/schemas/__init__.py so individual/group load
before bundle, since bundle.py now imports those classes at module load
time for the polymorphic_body registration.
Brings 534 commits of upstream onto the GIS feature branch. Resolved 14
conflicts; git absorbed the upstream spp_metrics_services -> spp_metric_service
module rename automatically.

Conflict resolutions:
- spp_api_v2_gis/__manifest__.py: use renamed deps spp_indicator/spp_analytics,
  keep branch additions spp_programs/spp_cel_domain/job_worker.
- spp_api_v2_gis/routers/statistics.py: keep branch's ProcessRegistry delegation,
  rename schema classes Statistic*->Indicator* (Statistic* no longer exported).
- spp_api_v2_gis/readme/DESCRIPTION.md: branch's OGC Processes wording + renamed
  spp.analytics.service.
- spp_hazard/{models,views,tests}: integrate CAP ingestion (branch) with upstream;
  list view uses severity_id (drop stale severity badge); union search fields.
- spp_hazard_programs/views: upstream's robust ribbon xpath.
- spp_mis_demo_v2/models/mis_demo_generator.py: upstream blueprint core + branch's
  opt-in volume/Luzon path (NEEDS REVIEW - see PR notes).
- spp_mis_demo_v2/data/demo_statistics.xml: branch's refactor (drop demographic
  stats, keep pwd_members) on live spp.indicator model.
- spp_demo/models/demo_stories.py: upstream household/district profile structure.
- spp/docker-compose.yml: upstream's jobworker service + branch CLI features.
- Use defined 'eligibility_manager' channel instead of bogus
  'root_program.eligibility_manager' (jobs were ignoring the concurrency limit).
- Exclude already-enrolled via get_beneficiaries() to match the base manager,
  instead of all program_membership_ids.
- Declare faker dependency on spp_mis_demo_v2 (volume generator path needs it).
…vice

Merge kept branch-added references to the pre-rename module/model names,
which broke install (spp_gis_report depended on the nonexistent
spp_metrics_services). Update module dep, demo XML external-id refs,
spp.metrics.* -> spp.metric.* model lookups, and docs.
- Guard against empty combined geometry (is_empty) before spatial query.
- Use Command.link instead of legacy (4, id) tuple in create wizard.
- Move DESCRIPTION.md to readme/ (OCA convention) and regenerate README.
- spp_hazard test_registrant: severity -> severity_id (CAP rename).
- spp_api_v2_gis process_registry + test_ogc_processes: spp.statistic ->
  spp.indicator (spp_statistic -> spp_indicator rename). Cleared 11 errors.
Copy link
Copy Markdown

@github-advanced-security github-advanced-security AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semgrep OSS found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

The branch's early HDX p-code areas (170 records replacing upstream's 28
named demo areas) were superseded by the dedicated spp_demo_phl_luzon module
(which loads its own area data). The leftover spp_demo changes broke upstream's
demo_stories + area_loader tests (missing NCR / named areas). Reverting
areas.xml, phl_curated.geojson, and a docstring in demo_data_generator.py to
origin/19.0; spp_demo_phl_luzon still provides the Luzon geodata.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants