Skip to content

feat(restrict): window a model's flattened matrices - #1036

Open
olivier-lacroix wants to merge 2 commits into
PyPSA:masterfrom
olivier-lacroix:feat/restrict-matrix-windows
Open

olivier-lacroix wants to merge 2 commits into
PyPSA:masterfrom
olivier-lacroix:feat/restrict-matrix-windows

Conversation

@olivier-lacroix

@olivier-lacroix olivier-lacroix commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Closes #768. Interested in having rolling horizon patterns in linopy.

  • I used the original proposition in terms of method naming. This can be changed to whatever people prefer.
  • My first attempt at this sliced each container individually. This allowed to work on the windowed model as on a normal model, but required a lot of code, and was kind of slowish.

Note

The following content was generated by AI.

What

Model.restrict returns a WindowModel backed by a slice of the parent's
flattened A, b, c, lb and ub. A window is a row slice plus a column
range; terms whose variables fall outside the window are folded into the RHS at
their actual matrix coefficient. No container is rebuilt.

window = model.restrict(snapshot=sns[:24], fix={"soc": initial})
window.solve(solver_name="highs")

for start in range(24, len(sns), 24):
    window.advance(snapshot=sns[start : start + 24], fix={"soc": initial})
    window.solve(solver_name="highs")
  • window.matrices exposes the window arrays, with the parent labels of its rows and columns.
  • window.solution and window.dual are Datasets, materialised per container on access, with column, row and objective scaling undone.
  • window.solve builds the native model from the window and updates it in place through ModelDiff.from_windows and each backend's existing _apply_* hooks. A backend without in-place updates rebuilds, and logs a warning; pass disallow_rebuild=True to raise instead.
  • window.advance re-slices the parent; a shape change rebuilds the solver.

Semantics

Limits

  • One selected dimension; several raise.
  • Every carrier of the selected dimension must contain the selected labels.
  • Indicators and quadratic objectives raise; native SOS raises; piecewise works; reformulated SOS still fails on its horizon-wide cardinality row.
  • A window is not a Model: it cannot take new constraints.

Benchmark

Build plus solver assembly, optimisation excluded. 35040 steps, a 1460-step
window, 24 chunks, three variables and three constraints.
hyperfine --warmup 1 --runs 5:

route wall peak RSS
full horizon 514.7 ms ± 7.4 202.6 MiB
chunked rebuild 1236.3 ms ± 11.1 170.2 MiB
restrict + advance 532.3 ms ± 4.6 190.7 MiB

Windowing costs 3.4% over one full build and avoids 24 rebuilds. In-process,
excluding import overhead: 0.170, 0.938 and 0.171 s.

Benchmark script
import sys
import time

import numpy as np
import pandas as pd

import linopy
from linopy.solvers import Solver

N, W = 35040, 1460
route = sys.argv[1] if len(sys.argv) > 1 else "full"


def build(index, prices):
    model = linopy.Model()
    x = model.add_variables(lower=0, coords=[index], name="x")
    y = model.add_variables(lower=0, upper=100, coords=[index], name="y")
    z = model.add_variables(lower=0, coords=[index], name="z")
    model.add_constraints(y - 0.99 * y.shift(time=1) - x + z == 0, name="balance")
    model.add_constraints(x + z <= 50, name="cap")
    model.add_constraints(z - 0.5 * x >= 0, name="mix")
    model.add_objective((prices * x + 0.1 * z).sum())
    return model


def assemble(model):
    Solver.from_name("highs", model=model, io_api="direct")


index = pd.RangeIndex(N, name="time")
prices = pd.Series(np.sin(np.arange(N) / 50), index=index)
fix = {"y": pd.Series(np.zeros(N), index=index)}

start = time.perf_counter()
if route == "full":
    assemble(build(index, prices))
elif route == "chunked":
    for offset in range(0, N, W):
        chunk = index[offset : offset + W]
        assemble(build(chunk, prices.loc[chunk]))
else:
    window = build(index, prices).restrict(time=index[:W], fix=fix)
    for offset in range(W, N, W):
        window.advance(time=index[offset : offset + W], fix=fix)
    assemble(window.as_model_view())

print(f"{route}: {time.perf_counter() - start:.3f}s")

Solver support

Only HiGHS has been exercised by hand. test_window_solve_across_solvers runs
the same recipe on highs, gurobi, cplex, scip, xpress and mosek, skipping
solvers that are absent, file-only, or installed without a license.

solve_window requires io_api="direct". A file-based backend would have to
write the window to disk on every solve, and has no in-place update.

Tests

test_restrict_matrix.py and test_windowing_integration.py, including a solver-parametrized run of the window recipe. linopy/restrict is at 100% statement and branch coverage.

AI assistance

Drafted with Claude Code: the implementation, tests and benchmark. Reviewed by hand before opening.

@olivier-lacroix
olivier-lacroix force-pushed the feat/restrict-matrix-windows branch 2 times, most recently from c48108a to 366d819 Compare October 20, 2026 11:23
@olivier-lacroix
olivier-lacroix marked this pull request as ready for review October 20, 2026 11:26
@codspeed

codspeed Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 181 untouched benchmarks
⏩ 181 skipped benchmarks1


Comparing olivier-lacroix:feat/restrict-matrix-windows (f78d4ca) with master (51c4977)

Open in CodSpeed

Footnotes

  1. 181 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@olivier-lacroix
olivier-lacroix force-pushed the feat/restrict-matrix-windows branch 3 times, most recently from 408ba31 to 3da2fd1 Compare October 20, 2026 11:40
@olivier-lacroix

olivier-lacroix commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll resolve typing issues

Edit: done

Olivier Lacroix and others added 2 commits October 25, 2026 14:41
`Model.restrict` returns a `WindowModel` backed by a slice of the parent's
`A`, `b`, `c`, `lb` and `ub` instead of a rebuild of every variable and
constraint. A window is a row slice plus a column range; boundary terms are
folded at their actual coefficient, and containers are materialised only when
their values are read.

- `WindowModel.matrices` exposes the window arrays; `solution`, `dual` and
  `objective` read them back with column, row and objective scaling undone.
- `WindowModel.solve` builds the native model from the window and moves it in
  place through `ModelDiff.from_windows` and the existing per-backend apply
  hooks.
- `advance` re-slices the parent; a shape change rebuilds the solver.
- A window that cuts a coupling without `fix` raises `RestrictionError`, naming
  the constraints.

Supported: one selected dimension, dense and CSR parents, masked rows.
Not supported: indicators, quadratic objectives, SOS, piecewise, several
selected dimensions, subset carriers.

`linopy/restrict` is at 100% statement and branch coverage.

Co-Authored-By: Claude Code <noreply@anthropic.com>
… test

Master removed the autouse semantics fixture with legacy semantics (PyPSA#1041).
The windowing equivalence test still requested it and errored at setup on
every platform.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@olivier-lacroix
olivier-lacroix force-pushed the feat/restrict-matrix-windows branch from eaa7222 to f78d4ca Compare October 25, 2026 03:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design: Model.restrict / advance — slice a model along a coordinate and condition on boundary values (rolling horizon)

1 participant