Test that dask collections can hold scipy.sparse arrays#3738
Test that dask collections can hold scipy.sparse arrays#3738mrocklin merged 1 commit intodask:masterfrom
Conversation
They can't do much with them because they don't support the ndarray interface, but it's useful for dask-ml work that we can produce them and then hand them off to dask.delayed or da.map_blocks workflows
|
Merging tomorrow morning if there are no comments. |
|
Thanks that looks good. Do you think it will help with #2842. I will try to have a look. |
|
Sorry I was confused #2842 was about the sparse.COO array (not scipy sparse matrix). I will have a deeper look at the |
|
No. Most Dask array operations still won't work. However I think that this will be useful for dask-ml workloads where we mostly call |
|
So dask array and dask dataframe can contain scipy sparse arrays well and we can apply functions across them, but the actual dask array and dask dataframe operations, like |
|
Yes I just tried to play a bit with it interactively and indeed it's already useful as it is and probably enough for the dask-ml use cases. |
They can't do much with them because they don't support the ndarray
interface, but it's useful for dask-ml work that we can produce them and
then hand them off to dask.delayed or da.map_blocks workflows
flake8 dask