Repository navigation
refactor: one Constraint class over a store-backed lhs and one netcdf layout (#1019 steps 4a-4c) - #1057
Conversation
…data derived (#1046) Constraint stores lhs, sign, rhs, labels, scaling, dual, binary fields and blocks; data is built from them. Store.positional_csr/canonical; CSRStore caches the positional CSR by weakref. add_constraints, to_constraint, netcdf reader and Model.copy construct from fields. Step 4a of #1019.
…Base/freeze API removed (#1047) A constraint in a sparse model is a Constraint whose lhs holds a CSRStore. update(), soften and set_blocks keep a sparse lhs (closes #1033). MatrixAccessor, dual write-back and Model.copy lose their isinstance branches. Twin-build test helpers deleted; tests parametrized over backends. Step 4b of #1019.
… refused (#1048) Store.to_netcdf_vars/from_netcdf_vars dispatch on a _store_kind attribute for constraint lhs, model expressions and the objective; the csr/dense legacy reader branches are deleted. read_netcdf refuses files without the 1.0 layout attribute. Round trips parametrized over both backends. Step 4c of #1019.
Docs no longer promise that pre-1.0 netcdf files load. read_netcdf tells a foreign file from a pre-1.0 one; missing store kind raises ValueError; layout attrs carry the _linopy_ prefix; dead order fallback removed.
Merging this PR will regress 2 benchmarks1 warning
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
…seStore _row_block sized buffers by term slots instead of nonzeros; DenseStore has_terms/take/terms copied broadcast term arrays; constraints_to_file held two slices; constraint construction copied const and coords. Peak memory of the export benchmarks is back to master level.
…in to_polars Store.isel returns views on DenseStore; Constraint.iterate_slices uses it instead of take, which copied coeffs and vars per LP slice.
… adds dims Reorder an incoming lhs to the constraint grid in Constraint._rebind so labels stay contiguous for range-based dual write-back; Store.transpose. Review nits: sizes reuse, dead _fill_value, Sequence[Hashable] annotation, release-note wording; tests for store swap dirtiness, masked duals on both backends, dense 1e-10 prune tolerance.
Closes #1046, closes #1047, closes #1048, closes #1033. Part of #1019 (steps 4a, 4b, 4c).
Note
The following content was generated by AI.
Changes proposed in this Pull Request
One
Constraintclass, one store behind it, one netcdf layout. Three steps of the #1019 plan in four commits:4a, f7eb1e1:
Constraintoverlhs. A constraint holds anlhs: LinearExpression(whose store is aDenseStoreorCSRStore) plussign,rhs,labels,scaling,dual,binary_var,binary_valandblocksas DataArrays on the lhs grid.datais derived.Store.canonical()andStore.positional_csr()join the store protocol; the positional CSR cache lives onCSRStore(keyed onlabel_to_posidentity and the active rows) and there is none onDenseStore. Every path that replaces the lhs store sets_coef_dirty, so the persistent snapshot and diff shortcuts keep working.4b, b7f7824: one
Constraintclass.CSRConstraint,ConstraintBaseand the freeze API are deleted. A constraint in a sparse model is aConstraintwhose lhs holds aCSRStore.update(),soften,set_blocksand thelocsetters keep a sparse lhs sparse (closes #1033).MatrixAccessor, the dual write-back andModel.copyhave noisinstancebranches on the store. The twin-build test helpers are gone; constraint tests are parametrized over thebackendfixture.4c, d75d04a: one netcdf layout. Constraint lhs, model expressions and the objective are written from the store through
Store.to_netcdf_vars()/from_netcdf_vars(), dispatching on a_linopy_store_kindattribute. Every store reads back as the kind it was written with. The_linopy_format="csr"branch, the dense-only reader and the legacy MultiIndex reader are deleted.read_netcdfrefuses files written by linopy < 1.0 with aValueError; a fixture file undertest/data/guards that.bd8be88: review fixes. Docs no longer promise that pre-1.0 files load; clearer read errors; layout attrs carry the
_linopy_prefix.Breaking changes (1.0)
CSRConstraint,ConstraintBase,Constraint.freeze(),Constraint.mutable(),add_constraints(freeze=)andConstraints.add(freeze=)are removed. UseModel(sparse=True),Constraint.to_sparse()/to_dense()andc.lhs.is_sparse.Constraint(lhs, sign, rhs, model, ...)replacesConstraint(data, model, ...);Constraint.from_datasetbuilds one from a Dataset. Writing intocon.datano longer changes the constraint.read_netcdf; rebuild and save them again.assert_conequalcompares the lhs withassert_equivalent, i.e. layout-free.Review and tests
Two review gates (after 4b, and full-branch before this PR) found no must-fix in the code; the surviving must-fixes were two stale doc statements, fixed in bd8be88. Full suite: 4747 passed, 84 skipped (
test/remoteexcluded, it needsgoogle-cloud-storage).mypy linopy,ruff check .and the doctests pass.Deferred review findings (nice-to-have, not blocking)
linopy/constraints.py(_FIELDS,__slots__,__init__,_bind,_fields).add_constraintsandadd_indicator_constraintsrepeat the same build block.Constraint.to_matrix_with_rhsduplicates the rhs/sense extraction inmatrices._row_block.MatrixAccessor.DenseStore.active_termsis public on one store only.from_datasetround-trip test is dense-only; the deep-copy test does not cover the field arrays;test_constraint_wrapped_methodsasserts nothing.label_rangereads back from netcdf as an ndarray, not a tuple (callers unpack it fine; the dense reader behaved the same before).objective.pyandio.py;read_netcdfreads scalar attrs leniently.to_netcdffails for expressions with tz-aware datetime, categorical or MultiIndex coords. Deserves its own issue.Checklist
AGENTS.md).doc.doc/release_notes.rstof the upcoming release is included.CI follow-ups (AI-generated)
5037b812: mypy over the test suite (CI runsmypy .).d9a03512,f535e29a: CodSpeed flagged a peak-memory regression (35 benchmarks, export up to 7x). Causes: export buffers sized by term slots instead of nonzeros,DenseStorecopying broadcast term arrays, per-slice copies in the LP writer. Fixed viaStore.nnzandStore.isel(views on the dense store). After the fix all large export benchmarks are at or below master (five improved, up to 7x); two tiny constraint-build ops (test_op[con_le_array],test_op[con_le_scalar]) stay about 34 KB above master, the fixed per-constraint cost of holding sign, rhs, scaling and labels as separate DataArrays. Residual on dense LP export: the int64 cell array fromStore.terms, about 8 bytes per nonzero; follow-up in Drop the per-nonzero cell array fromStore.terms()to cut transient memory in the dense LP export #1058.codecov/projectdrops 0.31 points to 88.54 % because about 2900 lines of well-covered code were deleted; patch coverage is 98 %.