Add mindeps build with all optional dependencies#10161
Add mindeps build with all optional dependencies#10161jrbourbeau merged 35 commits intodask:mainfrom
mindeps build with all optional dependencies#10161Conversation
charlesbluca
left a comment
There was a problem hiding this comment.
Most of the test failures have been resolved at this point, the only ones remaining I've left because I'm not sure if we'd prefer bumping the minimum supported version or just patching the tests with some compat code:
- the fastparquet failures are due to lack of support for reading/writing statistics & nullable dtypes
- the zarr failures are due to
zarr.Arrayexpecting an initialized store rather than a string or path-like
| - tblib=1.5.0 | ||
| - tiledb-py=0.8.1 | ||
| - xxhash=0.6.5 | ||
| - zarr=2.4.0 |
There was a problem hiding this comment.
Would you mind taking a look at #10137, since it's related?
There was a problem hiding this comment.
Yup! Left a comment there, in general looks like our options are either:
- setting the min version to 2.4.0 and modifying tests to explicitly instantiate zarr stores if we're using
<2.12.0 - setting the min version to 2.12.0 (~9 months old) like you've done in your PR
Do you have a preference here?
There was a problem hiding this comment.
I have a preference for setting 2.12.0 as the minimum
Co-authored-by: James Bourbeau <jrbourbeau@users.noreply.github.com>
jrbourbeau
left a comment
There was a problem hiding this comment.
Thanks @charlesbluca. This is looking really close
docs/source/install.rst
Outdated
| +-----------------+-------------+--------------------------------------------------------------------------------------------------------+ | ||
| | fastparquet | >=0.8.2 | Storing and reading data from Apache Parquet files | | ||
| +-----------------+-------------+--------------------------------------------------------------------------------------------------------+ | ||
| | gcsfs | >=0.4.0 | Storing and reading data located in Google Cloud Storage | |
There was a problem hiding this comment.
Worth noting here that gcsfs doesn't seem to be getting tested anywhere, so this min version is likely outdated - might be worth it to add some tests in the medium term, but I can see if we can bump this version based on our new fsspec min
jrbourbeau
left a comment
There was a problem hiding this comment.
This looks great, thanks for all your work here @charlesbluca
Adds a build running the test suite with all the optional dependencies used through Dask pinned to the lowest versions I could reasonably get working.
cc @jrbourbeau
mindepsbuild with all optional dependencies #10038pre-commit run --all-files