-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
HighlevelGraph refactor performance regressions #6732
Copy link
Copy link
Open
Labels
highlevelgraphIssues relating to HighLevelGraphs.Issues relating to HighLevelGraphs.needs attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.It's been a while since this was pushed on. Needs attention from the owner or a maintainer.
Description
https://pandas.pydata.org/speed/dask/#regressions?sort=1&dir=desc turns up some performance regressions related to the recent HLG refactor.
- https://pandas.pydata.org/speed/dask/#array.BlockInfoSingleton.time_optimize_singleton?python=3.7&commits=d4447e77
- https://pandas.pydata.org/speed/dask/#order.OrderRechunkTranspose.time_order_rechunk_transpose?python=3.7&commits=d4447e77
- https://pandas.pydata.org/speed/dask/#array.Blockwise.time_make_blockwise_graph?python=3.7&commits=d4447e77A
- https://pandas.pydata.org/speed/dask/#array.Slicing.time_slice_slice_head?python=3.7&commits=d4447e77 (partly fixed)
- https://pandas.pydata.org/speed/dask/#array.Slicing.time_slice_slice_tail?python=3.7&commits=d4447e77 (partly fixed)
- https://pandas.pydata.org/speed/dask/#array.BlockInfoSingleton.time_optimize_singleton?python=3.7&commits=d4447e77
- https://pandas.pydata.org/speed/dask/#array.Slicing.time_slice_int_tail?python=3.7&commits=d4447e77 (partly fixed)
- https://pandas.pydata.org/speed/dask/#array.Slicing.time_slice_int_head?python=3.7&commits=d4447e77 (partly fixed)
Some slowdown is likely to be expected, but I haven't taken the time to look into it yet. @madsbk or @jrbourbeau do you have a chance to glance through the benchmarks at https://github.com/dask/dask-benchmarks to see if any are concerning?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
highlevelgraphIssues relating to HighLevelGraphs.Issues relating to HighLevelGraphs.needs attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.It's been a while since this was pushed on. Needs attention from the owner or a maintainer.