-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
As a part of supporting units with Dask in xarray, work is being undertaken to implement the Dask collections interface on Pint Quantities to allow them to act as a "duck Dask Array" or "Dask Array-like" so that xarray can perform all its usual Dask-related functionality with a Pint Quantity in-between the xarray data structure and Dask Array (since, in the generally accepted typecasting heirachy, xarray wraps Pint and Pint wraps Dask, which allows for unit calculation at the construction rather than compute step).
Given that most if not all of xarray's current Dask support is special cased based on checking for instances of Dask Arrays, the question that has arisen is how should xarray check for a duck Dask Array? @crusaderky suggested asking the Dask team for a formalized answer that xarray (and really any other library that would have types above Dask Array on the type casting hierarchy) could implement, so I wish to do so here.
Ideas that have been brought up in pydata/xarray#4208 and other past xarray/pint issues include
- having a non-None
chunksattribute - numpy-array-like AND has a chunks method AND the chunks method returns a tuple
- Having specifically
__array_function__and__dask_graph__ - Having some amount of the Dask collection interface dunder methods (Support for duck Dask Arrays pydata/xarray#4208 (comment), Integration with Dask (add tests; implement the Dask collection interface on Quantity) hgrecco/pint#883 (comment))
xref pydata/xarray#4208
cc @shoyer, @crusaderky, @keewis, @rpmanser