perf: detect & fastpath no-op range slices in all layout types#3642
perf: detect & fastpath no-op range slices in all layout types#3642ianna merged 13 commits intoscikit-hep:mainfrom
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files
🚀 New features to boost your workflow:
|
|
The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR3642 |
|
@ianna I could add this fast path also to |
ianna
left a comment
There was a problem hiding this comment.
@pfackeldey - thanks! I wonder if moving the following check start == 0 and stop == self.length up to line 448 is all what is needed here.
ianna
left a comment
There was a problem hiding this comment.
@pfackeldey - looks good to me! Thanks! I'll enable auto merge.
This PR checks for trivial no-op slices in all layout types that essentially do nothing. If that's the case we can directly return
selfinstead of constructing new python classes (and loosing their cached_properties), doing plenty of unnecessary repetitive checks, etc.The most common operation that benefits from this improvement are
__setitem__calls on RecordArrays, see the following:Similar speedups are seen for VirtualNDArrays.
I expect this to speedup multiple other operations that traverse layouts as well.
Adding this to TypeTracerArrays is maybe possible (?) but much more complicated as the slices can be TypeTracerArrays themselves and thus need special & careful handling. I tried this, but the full logic turned out to be a bit complicated, and needs more thoughts -> maybe something for a future PR.
This improvement should become pretty noticeable in coffea analysis as it is highly common to store new fields on
events, and currently that's one of the most expensive operations - even more expensive than running awkward kernels. And that should not be the case given that this is a metadata only operation.This PR is reducing the total number of python allocations (new instances, etc) for a single
events["Jet", "pt"] = events["Jet", "pt"]setitem from8049to1692.