You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Drop the per-nonzero cell array from Store.terms() to cut transient memory in the dense LP export #1058
This issue was drafted by AI as a follow-up to #1057 (steps 4a-4c of #1019).
Describe the feature you'd like to see
Context
Store.terms() returns three arrays per live term: the flat cell index, the variable label and the coefficient (linopy/store.py, linopy/csr.py). The dense export (Constraint.to_polars in linopy/constraints.py, LinearExpression long-format helpers in linopy/expressions.py, assert_equivalent in linopy/testing.py) only uses the cell array to gather a per-cell value (constraint label or constant) onto the nonzeros. The int64 cell array is therefore a transient of about 8 bytes per nonzero that exists only to be used as an index and then dropped.
#1057 already sizes the export buffers by Store.nnz and slices dense stores by view (commits d9a0351, f535e29), so this cell array is the last per-nonzero residual in the dense LP path that is not part of the output.
Proposed direction
Change the protocol so the store returns the gathered value instead of the index, for example:
Store.terms(values: np.ndarray | None = None) returning (gathered, vars, coeffs), where gathered is values repeated per live term in that cell, or
Store.terms() returning (counts, vars, coeffs) with one int per cell, so callers expand with np.repeat(values, counts).
Either way the per-nonzero index never materialises. DenseStore can derive counts from the keep mask with keep.reshape(-1, nterm).sum(1); CSRStore already has indptr. Call sites to adapt: Constraint.to_polars, LinearExpression.to_polars/long-format helpers and linear_terms, assert_equivalent.
Separate from this: two CodSpeed memory benchmarks (test_op[con_le_array], test_op[con_le_scalar]) sit about 34 KB above master after #1057. That is the fixed cost of the field DataArrays on Constraint, not the cell array, and is out of scope here.
Acceptance criteria
Both backends implement the new signature; the conformance test in test/test_store_conformance.py covers it.
to_polars, LP writing and assert_equivalent produce identical output on dense and sparse stores.
CodSpeed memory benchmarks benchmarks/drivers/test_to_lp.py and benchmarks/drivers/test_to_solver.py show the peak drop for dense models; no regression for CSR.
Activity