-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Support for sharding when storing dask arrays to zarr #11778
Copy link
Copy link
Closed
Labels
arrayenhancementImprove existing functionality or make things work betterImprove existing functionality or make things work betterioneeds attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.It's been a while since this was pushed on. Needs attention from the owner or a maintainer.
Metadata
Metadata
Assignees
Labels
arrayenhancementImprove existing functionality or make things work betterImprove existing functionality or make things work betterioneeds attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.It's been a while since this was pushed on. Needs attention from the owner or a maintainer.
Hi,
With the release of zarr-python 3, the feature to store multiple chunks in a single storage object (i.e. sharding) was introduced. Since, Dask has started to support zarr-python 3, I would request the option to specify the the shards in the
to_zarr()function be introduced. This will be a very helpful feature for people trying to store large arrays on regular file systems while needing fast random reads.