Relax dependency constraints#1713
Open
filippsatverily wants to merge 8 commits intocdisc-org:mainfrom
Open
Conversation
Moves dependency constraints to pyproject.toml. Makes requirements.txt a lockfile.
Fixes an incompatibility caused by click 8.3.0, which passes the default value as-is.
Fixes an incompatibility caused by pyreadstat 1.2.9, which changed original_variable_type from 'NULL' to None
Works around an behavior change in jsonpath-ng 1.8.0 where Child.str gets wrapped in parenthesis.
Fixes tokenization errors when using dask 2024.8.1+. Starting with this version, dask enforces that tokens remain stable across pickle round-trips (dask/dask#11320). Capturing self in a lambda fails this check because instance objects can have non-deterministic pickle representations. Since calculate_variable_value_length is already a static method, replacing self with the class name is enough to remove the capture.
Dask 2025.4.0 optimizes multiple DataFrames together, which exposes division mismatches and causes dask to throw an error. This change removes a source of repartitioning, preserving the divisions when assigning a pandas series to a dask dataframe
Fixes a unit test to support pandas 2.2.0+. The pandas release fixes an sorting bug with pandas-dev/pandas#54611. This commit changes the expected results accordingly.
filippsatverily
force-pushed
the
filipps/update_deps_main
branch
from
14ffff8 to
a646ccf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary: expand dependency constraints and turn requirements.txt from a requirements file into a lockfile. This will allow the PyPi version of CORE to be used as a library in a larger project where other dependency constraints exist.
List of changes:
pyproject.tomldefaults that clash withrequiredto support a behavior change/fix in click 8.3.0DatasetXPTMetadataReader.readto support a behavior change in pyreadstat 1.2.9USDMDataService.__get_full_pathto support a behavior change in jsonpath-ng 1.8.0selfcapture inContentsDefineVLMDatasetBuilderto support dask 2024.8.1__setitem__reindexing inDaskDatasetto fix errors surfaced in dask 2025.4.0test_dataset_metadata_define_dataset_builderto support a sort behavior fix in pandas 2.2.0Tested scenarios: