Handle legacy nested datasets (with tests)#850
Handle legacy nested datasets (with tests)#850joshmoore merged 7 commits intozarr-developers:masterfrom
Conversation
Each fixture (flat & nested) should have a version which does not include any new metadata.
|
Hello @joshmoore! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found: There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2021-10-19 19:48:47 UTC |
Codecov Report
@@ Coverage Diff @@
## master #850 +/- ##
=======================================
Coverage 99.94% 99.94%
=======================================
Files 31 31
Lines 10936 10952 +16
=======================================
+ Hits 10930 10946 +16
Misses 6 6
|
|
I'd propose to get this out as a quick 2.10.2 if there are no objections. |
|
Thanks Josh! 😄 |
|
Note that one of the tests that passed here is failing in the feedstock: |
1 similar comment
|
Note that one of the tests that passed here is failing in the feedstock: |
| run: | | ||
| conda activate minimal | ||
| python -m pip install -e . | ||
| python -m pip install . |
There was a problem hiding this comment.
Curious, why this was needed? Did it fail without this? If so, that may be indicative of the issue we are seeing in conda-forge
There was a problem hiding this comment.
Dropping the editable install fixed the error "No module named 'pip'". More info in pypa/pip#10573
fix #840
As pointed out @abred, zarr 2.10 (and earlier) was no longer properly handling the opening of nested datasets if they did not contain the
dimension_separatormetadata in.zarray. This is fixed with this PR. However, there are a number of cases which still xfail:Essentially for legacy datasets without the
dimension_separatorthe following holds:NestedDirectoryStore.NestedDirectoryStore.These restrictions can be circumvented by updating your .zarray files to contain the proper metadata:
TODO: