-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Desired functionality is something like the following in the 1d case
>>> dask_slice('y', 'x', (100,), [(20, 20, 20, 20, 20)], ([5, 1, 27],)) # x[[5, 1, 27]]
{('x', 0): (np.concatenate, [(getitem, ('x', 0), [5, 1]),
(getitem, ('x', 1), [7])])}Though I suspect that there are other complications.
One is if we index with a list with elements that don't access blocks consecutively as in [5, 27, 1]. In this case we'll maybe want to slice each block only once and then reshuffle the results. This could probably be done by sorting the input list [5, 27, 1] -> [1, 5, 27] then doing the concatenate and finally doing another getitem on top of that to reorder the indices (in this case. (getitem, ..., [2, 0, 1])).
I'm also uncertain if this will be much more complex in n-dimensions when we mix with integers and slices.
CCing @nevermindewe in case he has ideas here.
One motivation for this is to support nd-grouping applications as is done in xray (cc @shoyer)