Dask contains parallel algorithms for parallel arrays based on the numpy syntax. While commonly used with numpy arrays they also work with other libraries that mimic subsets of the numpy ndarray API.

The CuPy library implements a NumPy API compatible GPU ndarray. It would be interesting to see how these two libraries interact, where performance bottlenecks are, and under what conditions they may be effectively used together.

To experiment with this one would want to do a few things:

1. Register cupy operations for concatenate and tensordot. Examples doing this for the sparse library are at the top of dask/array/core.py
2. Create a GPU array using da.from_array, or da.from_delayed and da.concatenate/stack. See http://dask.pydata.org/en/latest/array-creation.html
3. If you want to use distributed computing then ensure that cupy ndarrays can be efficiently serialized, probably by implementing __setstate__ and __getstate__ methods for them if they don't already exist.

This should be sufficient to get something up and running.