Description

This PR depends on dask/dask#9475 (Now Merged)

After dask#9475, external libraries are now able to implement (and expose) their own DataFrameBackendEntrypoint definitions to specify custom creation functions for DataFrame collections. This PR introduces the CudfBackendEntrypoint class to create dask_cudf.DataFrame collections using the dask.dataframe API. By installing dask_cudf with this entrypoint definition in place, you get the following behavior in dask.dataframe:

import dask.dataframe as dd
import dask

# Tell Dask that you want to create DataFrame collections
# with the "cudf" backend (for supported creation functions).
# This can also be used in a context, or set in a yaml file
dask.config.set({"dataframe.backend": "cudf"})

ddf = dd.from_dict({"a": range(10)}, npartitions=2)
type(ddf)  # dask_cudf.core.DataFrame

Note that the code snippet above does not require an explicit import of cudf or dask_cudf. The following creation functions will support backend dispatching after dask#9475:

• from_dict
• read_paquet
• read_json
• read_orc
• read_csv
• read_hdf

See also: dask/design-docs#1

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.