fix: add_variables ignoring coords for DataArray bounds#614
fix: add_variables ignoring coords for DataArray bounds#614FabianHofmann merged 14 commits intomasterfrom
Conversation
When DataArray bounds were passed to add_variables with explicit coords, the coords parameter was silently ignored because as_dataarray skips conversion for DataArray inputs. Now validates DataArray bounds against coords: raises ValueError on mismatched or extra dimensions, and broadcasts missing dimensions via expand_dims. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Test MultiIndex coords (validation skip), xarray Coordinates object, dims-only DataArrays, and upper bound mismatch detection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Infer dims from dict keys when dims is None and the input is a scalar. Previously this raised xarray's CoordinateValidationError because xarray can't broadcast a 0-dim value to coords without explicit dims. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate add_variables tests into TestAddVariablesBoundsWithCoords class with parameterized tests covering all bound types (scalar, np.number, numpy, pandas, list, DataArray, DataArray-no-coords) x both coord formats (sequence, dict). Also fixes as_dataarray for scalars with dict coords by inferring dims from dict keys. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Test DataArray+numpy, DataArray+scalar, DataArray+DataArray combos for lower/upper. Also test both bounds covering different dim subsets with broadcast, and that only the mismatched bound raises ValueError. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add numpy-0d and dataarray-0d to the parameterized bound type tests. Fix numpy_to_dataarray to infer dims from dict keys for 0-dim arrays, matching the scalar fix in as_dataarray. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cover three gaps: coords inferred from bounds (no coords arg) for DataArray and pandas, multi-dimensional coord specifications with both scalar and DataArray bounds, and real-world coordinate types (string regions, datetime index) including mismatch detection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verify lower(time, space) and upper(space, time) align correctly via xarray broadcast. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a DataArray bound has the same coordinate values as coords but in a different order, reindex to match instead of raising ValueError. Still raises when the values actually differ (not just reordered). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unreachable hasattr(coords, "dims") branch in _coords_to_dict (xarray Coordinates are Mappings, caught by isinstance check above). Add Any type annotations to parameterized test arguments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@FabianHofmann Sorry for all the branches. I really wanted to make the fix more universal and establish a distinction between converting to a dataarray and converting to a dataarray and validate, but i hot lost in the process. |
Pin to the fix/add-variables-dataarray-coords branch which fixes DataArray coords handling in add_variables. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
very good! thanks as always @FBumann ; the tests look very sensible. I think they functionally overlap a little with the tests in test_variable.py where they should be moved as well. So I would be fine in keeping them but move them there. otherwise I see this ready to merge
| # -- Reordered coordinates --------------------------------------------- | ||
|
|
||
| def test_reordered_coords_reindexed(self, model: "Model") -> None: | ||
| """Same coord values in different order should reindex, not raise.""" |
There was a problem hiding this comment.
I guess this is something we need to keep in mind for #591 where this should be disallowed
Per review feedback, these tests belong in test_variable.py where they overlap with existing variable tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
@FabianHofmann all set to merge |
* docs: restructure upcoming release notes and fold in missing PRs Group the upcoming version block into Features / Performance / Bug Fixes / Breaking Changes / Documentation sections so the headline (piecewise) leads, and add the entries for #589, #595, #601, #614, #619, #635, #656, #671, #672, #674. Tighten the piecewise block to its final state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: tighten upcoming changelog and drop internal-only entries Trim verbose phrasing in the piecewise / variables / model / solvers sections, fold subset-superset sub-bullets into one paragraph, and drop two entries that aren't user-facing for a release notes audience: sphinx-copybutton (doc tooling) and Model.__weakref__ (only relevant to extension authors). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: move align convention from breakpoints() to Slopes in changelog #673 removed the slopes-mode (and slopes_align kwarg) from breakpoints(); the align kwarg now lives on the Slopes class. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: move SOS reformulation bullet from Variables to Model SOS reformulation is a model-rewrite/solve-pipeline concern, not a variable attribute. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: split coord alignment into Expressions, move CPLEX to Bug Fixes - New *Expressions* subsection holds the subset/superset coord harmonization, which was misfiled under *Model*. - CPLEX quality-attribute handling is a fix for crashes on missing attributes, not a new feature — moved to **Bug Fixes**. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: fold as_dataarray MultiIndex fix into add_variables bullet #659 fixes a regression introduced by #614 in the same release cycle — no end user ever saw the broken state, so a standalone bullet overstates the change. Net behavior is captured by extending the add_variables bullet to mention MultiIndex coords. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: tighter pass on upcoming changelog Drop implementation details that belong in API docs (numpy-vs-pandas note, JSON encoding for netCDF, "with no auxiliary variables" piecewise detail), merge the two OETC bullets, and trim "Add X. Supports Y." wrappers across most lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: rephrase active gating bullet to avoid output-zeroing implication Previous wording ("zeros all auxiliaries when off") was true at the auxiliary level but glossed over the bounded-tuple case where the output is not automatically pinned to 0. Drop the implication and defer the detail to the docstring. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: drop option-name detail from upcoming changelog Trim references to specific kwargs/attributes the reader doesn't need in the high-level summary: method="auto" parens, align="pieces|leading", deep / include_solution, reformulate_sos="auto", solver_name / **solver_options, max_dual_infeasibility example, and the operator-by-operator coord-alignment breakdown. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
Two bugs in
add_variableswhen bounds are DataArrays andcoordsis provided. Bothlowerandupperare affected identically.This is a focused extract from #551 — bug fixes only, no
as_dataarrayrefactoring.Bug 1: DataArray coords silently ignored → NaN bounds
The fix validates DataArray bounds against
coords: mismatched or extra dims raiseValueError, missing dims are broadcast viaexpand_dims, reordered coords (same values) are reindexed.Bug 2: Dict coords crash for all input types
Fixed by inferring
dimsfrom dict keys inas_dataarrayandnumpy_to_dataarrayfor scalar/0-dim inputs.Changes
linopy/model.py_validate_dataarray_bounds+_coords_to_dicthelpers, called inadd_variableslinopy/common.pydimsfrom dict keys for scalars/0-dim arraystest/test_common.pyNo existing tests were modified or removed — all changes to
test/test_common.pyare purely additive. All 32 existing test functions from master are preserved.Test results: master vs this branch
CoordinateValidationError)ValueErrorValueErrorFull test suite: 2570+ passed, 236 skipped.
🤖 Generated with Claude Code