Skip to content

Method to make uniformly chunked Dask Arrays #3302

@jakirkham

Description

@jakirkham

In the process of applying some operations on Dask Arrays, chunk sizes can become a bit heterogeneous. However storage applications tend to prefer more homogeneous chunk sizes. While it is possible to the store a Dask Array to a storage format with different chunk sizes, this typically comes at a penalty of needing locking. Certainly one can rechunk the Dask Array manually. However it would be nice to have a method for Dask Arrays, which has some better ideas as to what sort of rechunking will come with low penalty. This method can then determine some reasonable homogeneous chunk size and rechunk the Dask Array using them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions