Add test for map overlap ordering#3652
Conversation
af0c116 to
3ddc209
Compare
3ddc209 to
d83dd6c
Compare
|
cc also @TomAugspurger |
|
Actually, as discussed in the original issue, this does seem to have a significant effect in 2d |
|
Seems reasonable (as you probably know, this doesn't help out with the ordering issue surfaced in dask-ml). |
|
Interesting, thanks for sharing Matt. Will give this a closer look. Hmm...does it have negligible impact Tom or does it make things worse? As you can imagine, we care about optimizing both of these cases as we do use both. :) |
|
AFAICT, it has no effect on the other case. |
|
From a first principles point of view, this will only improve things, however it may add to the cost of calling order. |
|
OK, I tested the increased number of steps through order due to this change by looking at |
|
Thanks @mrocklin :) |
….com/convexset/dask into fix-tsqr-case-chunk-with-zero-height * 'fix-tsqr-case-chunk-with-zero-height' of https://github.com/convexset/dask: fixed typo in documentation and improved clarity Implement .blocks accessor (dask#3689) Fix wrong names (dask#3695) Adds endpoint and retstep support for linspace (dask#3675) Add the @ operator to the delayed objects (dask#3691) Align auto chunks to provided chunks, rather than shape (dask#3679) Adds quotes to source pip install (dask#3678) Prefer end-tasks with low numbers of dependencies when ordering (dask#3588) Reimplement argtopk to release the GIL (dask#3610) Note `da.pad` can be used with `map_overlap` (dask#3672) Allow tasks back onto ordering stack if they have one dependency (dask#3652) Fix extra progressbar (dask#3669) Break apart uneven array-of-int slicing to separate chunks (dask#3648) fix for `dask.array.linalg.tsqr` fails tests (intermittently) with arrays of uncertain dimensions (dask#3662)
See #3554
cc @jakirkham
This doesn't resolve the issue in 2d though, and may be just a hack.
flake8 dask