006 download rinf topology

.github/agents/copilot-instructions.md

@@ -9,6 +9,8 @@ Auto-generated from all feature plans. Last updated: 2026-01-09
 - File-based I/O (CSV / GeoJSON); no DB. R-tree (`rstar`) in-memory spatial index reused for coordinate resolution. (004-train-detections)
 - Rust 1.75 (tp-net crate) + C# 12 / .NET 8 (TpLib managed) + `csbindgen` (FFI stub generation), `serde_json` (FFI marshalling), `tp-lib-core` (core algorithms); C# side: `System.Text.Json` (deserialization), xUnit (testing) (005-dotnet-bindings)
 - N/A — stateless function calls only (005-dotnet-bindings)
+- Rust 2021 workspace (`tp-core`, `tp-cli`, `tp-py`, `tp-net`) + Python bindings via `pyo3` + C# 12 / .NET 8 bindings + Existing workspace crates (`geo`, `geojson`, `chrono`, `serde`, `serde_json`, `clap`, `pyo3`, `csbindgen`) plus an HTTPS client for SPARQL access (`ureq ` blocking client with JSON response handling) (006-download-rinf-topology)
+- N/A for persistent storage; per-run in-memory retrieval/validation only (006-download-rinf-topology)
 
 - Rust 1.75+ (edition 2021) (002-train-path-calculation)
 
@@ -43,9 +45,9 @@ For Python source files (`.py`) changed under `tp-py/`:
 Rust 1.75+ (edition 2021): Follow standard conventions
 
 ## Recent Changes
+- 006-download-rinf-topology: Added Rust 2021 workspace (`tp-core`, `tp-cli`, `tp-py`, `tp-net`) + Python bindings via `pyo3` + C# 12 / .NET 8 bindings + Existing workspace crates (`geo`, `geojson`, `chrono`, `serde`, `serde_json`, `clap`, `pyo3`, `csbindgen`) plus an HTTPS client for SPARQL access (`reqwest` blocking client with JSON response handling)
 - 005-dotnet-bindings: Added Rust 1.75 (tp-net crate) + C# 12 / .NET 8 (TpLib managed) + `csbindgen` (FFI stub generation), `serde_json` (FFI marshalling), `tp-lib-core` (core algorithms); C# side: `System.Text.Json` (deserialization), xUnit (testing)
 - 004-train-detections: Added Rust 1.91.1+ (workspace edition 2021) + `geo` 0.28, `rstar` 0.12, `geojson` 0.24, `csv` 1.x, `serde`/`serde_json`, `chrono` (DateTime<FixedOffset>), `petgraph`, `proj4rs` 0.1.9; webapp: `axum`, `tokio`, Leaflet (static)
-- 003-path-review-webapp: Added Rust 2021 edition, latest stable (1.80+)
 
 
 <!-- MANUAL ADDITIONS START -->

.github/workflows/ci.yml

@@ -82,11 +82,14 @@ jobs:
           maturin develop
 
       - name: Install test dependencies
-        run: .venv/bin/pip install pytest
+        run: .venv/bin/pip install pytest ruff
 
       - name: Run Python tests
         run: .venv/bin/pytest tp-py/python/tests/
 
+      - name: Run ruff
+        run: .venv/bin/ruff check tp-py/python
+
   dotnet:
     name: .NET Tests
     runs-on: ubuntu-latest

Cargo.toml

@@ -64,6 +64,10 @@ tokio = { version = "1", features = ["full"] }
 rust-embed = { version = "8", features = ["debug-embed"] }
 open = "5"
 
+# HTTP client (RINF SPARQL retrieval — feature 006)
+ureq = { version = "2.10", features = ["json"] }
+url = "2.5"
+
 [profile.dev]
 debug = 2
 opt-level = 0

README.md

@@ -19,10 +19,11 @@ Train positioning library excels in post-processing the GNSS positions of your m
 - 🛤️ **Train Path Calculation**: Probabilistic path calculation through rail networks using topology
 - 🗺️ **Interactive Path Review**: Browser-based map webapp to visually review and edit calculated paths before projection
 - 🌍 **CRS Aware**: Explicit coordinate reference system handling (EPSG codes)
-- ⏰ **Timezone Support**: RFC3339 timestamps with explicit timezone offsets; timezone-less ISO 8601 datetimes assumed UTC
+- ⏰ **Timezone Support**: RFC3339 timestamps with explicit timezone offsets; naive (timezone-less) ISO 8601 datetimes are accepted on input and assumed to be in the host's **local** timezone. All emitted timestamps include an explicit timezone offset.
 - 📊 **Multiple Formats**: CSV and GeoJSON input/output
 - 🧪 **Well Tested**: 460 comprehensive tests (all passing) - unit, integration, contract, CLI, and doctests
 - ⚡ **Production Ready**: Full CLI interface with validation and error handling
+- 🌐 **Automatic RINF Retrieval**: When a topology file is omitted, the library can download a bounding-box subset of the [ERA RINF](https://data-interop.era.europa.eu/) network on demand
 
 ## Train Path Calculation
 
@@ -55,6 +56,12 @@ tp-cli --gnss positions.csv --network network.geojson --output result.csv --revi
 
 # Launch standalone webapp to review/edit a path file
 tp-cli webapp --network network.geojson --train-path path.csv --output reviewed_path.csv
+
+# Fetch the RINF topology covering a GNSS file (inspection helper).
+# Writes the retrieved netelements + netrelations to GeoJSON even when
+# validation reports issues (e.g. coarse geometries), then exits with the
+# corresponding non-zero status code.
+tp-cli fetch-topology --gnss positions.geojson --output topology.geojson
 ```
 
 ### Debug Output
@@ -295,6 +302,28 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
 }
 ```
 
+## Automatic RINF Topology Retrieval
+
+When you do not supply a `network` file, `tp-lib` can derive the bounding box from
+your GNSS input (and optional path) and download a fresh subset of the ERA RINF
+topology from a SPARQL endpoint.
+
+- **Default endpoint**: `https://graph.data.era.europa.eu/repositories/rinf-plus`
+- **Default buffer**: 1000 m around the GNSS extent
+- **Default timeout**: 60 seconds per HTTP request
+- **Coarse-geometry warning threshold**: netelements longer than 250 m without a WKT geometry
+
+Outcome categories (mapped to typed errors / exit codes across all bindings):
+
+| Category | CLI exit code | .NET exception | Python exception |
+|---|---|---|---|
+| `invalid_gnss_input` | 4 | `TpLibInvalidGnssInputException` | `InvalidGnssInputError` |
+| `rinf_missing_coverage` | 5 | `TpLibRinfMissingCoverageException` | `RinfMissingCoverageError` |
+| `rinf_incomplete_topology` | 6 | `TpLibRinfIncompleteTopologyException` | `RinfIncompleteTopologyError` |
+| `rinf_retrieval_failed` | 7 | `TpLibRinfRetrievalFailedException` | `RinfRetrievalFailedError` |
+
+See per-language READMEs (`tp-cli/`, `tp-py/`, `tp-net/`) for end-to-end examples.
+
 ## Implementation Notes
 
 ### Performance
@@ -313,7 +342,7 @@ latitude,longitude,timestamp,altitude,hdop
 50.8503,4.3517,2025-12-09T14:30:00+01:00,100.0,2.0
 ```
 
-- RFC3339 timestamps with timezone (e.g. `2025-12-09T14:30:00+01:00` or `2025-12-09T14:30:00Z`); timezone-less ISO 8601 datetimes (e.g. `2025-12-09T14:30:00`) are accepted and assumed UTC
+- RFC3339 timestamps with timezone (e.g. `2025-12-09T14:30:00+01:00` or `2025-12-09T14:30:00Z`); naive ISO 8601 datetimes without timezone (e.g. `2025-12-09T14:30:00` or `2025-12-09 14:30:00`) are accepted on input and interpreted in the host's **local** timezone. All output timestamps are emitted in RFC3339 form with an explicit timezone offset.
 - CRS must be specified via `--crs` flag
 - Column names configurable with `--lat-col`, `--lon-col`, `--time-col`
 

deny.toml

@@ -100,12 +100,13 @@ allow = [
     "ISC",
     "BSD-2-Clause",
     "BSD-3-Clause",
-    "BSL-1.0",           # Boost Software License (permissive)
-    "CC0-1.0",           # Creative Commons Zero (public domain)
-    "Unlicense",         # Public domain
-    "Zlib",              # zlib License (permissive)
-    "Unicode-DFS-2016",  # Unicode License (for Unicode data)
-    "Unicode-3.0",       # Unicode License v3 (for Unicode data, used by unicode-ident)
+    "BSL-1.0",              # Boost Software License (permissive)
+    "CC0-1.0",              # Creative Commons Zero (public domain)
+    "Unlicense",            # Public domain
+    "Zlib",                 # zlib License (permissive)
+    "Unicode-DFS-2016",     # Unicode License (for Unicode data)
+    "Unicode-3.0",          # Unicode License v3 (for Unicode data, used by unicode-ident)
+    "CDLA-Permissive-2.0",  # Community Data License Agreement Permissive 2.0 (permissive)
 ]
 # The confidence threshold for detecting a license from license text
 confidence-threshold = 0.8

specs/006-download-rinf-topology/checklists/requirements.md

@@ -0,0 +1,35 @@
+# Specification Quality Checklist: ERA RINF Network Download
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning  
+**Created**: 2026-05-13  
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- Validation passed on first review.
+- External source behavior is specified in outcome terms only; technical query design is intentionally deferred to planning.

specs/006-download-rinf-topology/contracts/api.md

@@ -0,0 +1,194 @@
+# API Contracts: ERA RINF Topology Retrieval
+
+**Phase**: Phase 1 — Design & Contracts  
+**Date**: 2026-05-13  
+**Feature**: `006-download-rinf-topology`
+
+This document specifies the public contracts for automatic topology retrieval across the external SPARQL boundary and the repo's user-facing integration surfaces.
+
+---
+
+## 1. External SPARQL Contract
+
+### Endpoint
+
+- `POST https://graph.data.era.europa.eu/repositories/rinf-plus`
+- Request content type: `application/sparql-query` or standard form-encoded `query=` payload
+- Response content type: `application/sparql-results+json`
+
+### Query A: Netelements by Search Polygon
+
+**Inputs**:
+- `search_polygon_wkt: string` in WGS84 lon/lat order
+
+**Result columns**:
+- `netelement`
+- `netelement_wkt`
+
+**Contract requirements**:
+- Returns every `era:LinearElement` whose geometry intersects the search polygon.
+- Returned `netelement_wkt` must be parseable into a LineString.
+
+### Query B: Netrelations by Retrieved Elements
+
+**Inputs**:
+- `seed element IRIs` from Query A
+- current date for validity filtering
+
+**Result columns**:
+- `netrelation`
+- `netelementA`
+- `netelementB`
+- `isOnOriginOfElementA`
+- `isOnOriginOfElementB`
+- `navigability`
+
+**Contract requirements**:
+- Only currently valid netrelations are returned.
+- Each row must reference two netelements that can be mapped into the retrieved topology bundle.
+
+---
+
+## 2. CLI Contract (`tp-cli`)
+
+Topology-dependent commands continue to support manual topology input and gain automatic retrieval when `--network` is omitted.
+
+### Default / `calculate-path`
+
+```text
+tp-cli calculate-path --gnss <FILE> [--network <FILE>] [--rinf-endpoint <URL>] [--rinf-buffer-meters <N>]
+```
+
+**Behavior**:
+- If `--network` is provided, CLI uses supplied topology and does not contact RINF.
+- If `--network` is omitted, CLI derives a search polygon from GNSS, retrieves RINF topology, validates it, and then runs path calculation.
+
+**Failure outcomes**:
+- Invalid GNSS input: non-zero exit, stderr explains GNSS input is empty/invalid.
+- Missing coverage: non-zero exit, stderr explains no topology was available for the area.
+- Incomplete topology: non-zero exit, stderr explains coarse geometry or missing netrelations.
+- Endpoint failure: non-zero exit, stderr explains retrieval failed upstream.
+
+### `simple-projection` and other topology-dependent commands
+
+Same source-selection rule applies: omit `--network` to trigger automatic RINF retrieval.
+
+---
+
+## 3. Python Binding Contract (`tp-py`)
+
+### Retrieval options class
+
+```python
+class RinfRetrievalOptions:
+    endpoint_url: str = "https://graph.data.era.europa.eu/repositories/rinf-plus"
+    buffer_meters: float = 1000.0
+```
+
+### Projection
+
+```python
+project_gnss(
+    gnss_file: str,
+    gnss_crs: str,
+    network_file: str | None = None,
+    network_crs: str | None = None,
+    target_crs: str | None = None,
+    config: ProjectionConfig | None = None,
+    rinf_options: RinfRetrievalOptions | None = None,
+)
+```
+
+### Path calculation
+
+```python
+calculate_train_path(
+    gnss_file: str,
+    gnss_crs: str,
+    network_file: str | None = None,
+    network_crs: str | None = None,
+    config: PathConfig | None = None,
+    rinf_options: RinfRetrievalOptions | None = None,
+)
+```
+
+### Detections preparation
+
+```python
+prepare_detections(
+    gnss_file: str,
+    detections_file: str,
+    network_file: str | None = None,
+    rinf_options: RinfRetrievalOptions | None = None,
+)
+```
+
+**Behavior**:
+- `network_file is not None`: supplied topology is authoritative; no retrieval.
+- `network_file is None`: bindings invoke Rust retrieval/validation logic using `rinf_options` (or defaults).
+- Missing coverage and endpoint failures surface as typed Python exceptions
+  (`InvalidGnssInputError`, `RinfMissingCoverageError`,
+  `RinfIncompleteTopologyError`, `RinfRetrievalFailedError`).
+
+---
+
+## 4. .NET Contract (`tp-net`)
+
+Because C# overload resolution cannot distinguish overloads that differ only
+in nullable-reference annotations, the auto-retrieval entry points use the
+`*Auto` suffix.
+
+### Projection
+
+```csharp
+public static IReadOnlyList<ProjectedPosition> Projection.ProjectGnssAuto(
+    NetworkInput? network,
+    GnssInput gnss,
+    ProjectionConfig? config = null,
+    RinfRetrievalOptions? rinfOptions = null);
+```
+
+### Path calculation
+
+```csharp
+public static PathResult PathCalculation.CalculateTrainPathAuto(
+    NetworkInput? network,
+    GnssInput gnss,
+    PathConfig? config = null,
+    PreparedDetections? detections = null,
+    RinfRetrievalOptions? rinfOptions = null);
+```
+
+### Retrieval options type
+
+```csharp
+public sealed class RinfRetrievalOptions
+{
+    public string EndpointUrl { get; set; } = "https://graph.data.era.europa.eu/repositories/rinf-plus";
+    public double BufferMeters { get; set; } = 1000.0;
+}
+```
+
+**Behavior**:
+- `network != null`: no RINF retrieval is attempted.
+- `network == null`: the wrapper calls the Rust retrieval workflow before
+  invoking the existing algorithms.
+- Failures surface as distinct typed exceptions:
+  `TpLibInvalidGnssInputException`, `TpLibRinfMissingCoverageException`,
+  `TpLibRinfIncompleteTopologyException`, `TpLibRinfRetrievalFailedException`.
+
+---
+
+## 5. Shared Outcome Contract
+
+All surfaces must preserve the same semantic outcome categories:
+
+| Outcome | Meaning |
+|---|---|
+| `success` | Topology was supplied or retrieved and validated successfully |
+| `invalid_input` | GNSS data was empty or unusable before any retrieval attempt |
+| `missing_coverage` | No suitable topology could be retrieved for the search region |
+| `incomplete_topology` | Retrieved topology failed validation, including coarse geometry or zero netrelations |
+| `endpoint_failure` | The external SPARQL request failed or returned an unusable response |
+
+No interface is allowed to collapse these categories into a generic failure message.