This repository was archived by the owner on Feb 26, 2025. It is now read-only.
backport: Optimize chained hyperslab selection. #1032
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7bd8150 to
6e7b550
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## v2.x #1032 +/- ##
==========================================
+ Coverage 86.13% 86.24% +0.10%
==========================================
Files 90 90
Lines 5858 5902 +44
==========================================
+ Hits 5046 5090 +44
Misses 812 812 ☔ View full report in Codecov by Sentry. |
6e7b550 to
a7b22df
Compare
A common pattern for creating semi-unstructured selection is to use many
(small) RegularHyperSlab and chain them:
```
HyperSlab hyperslab;
for(auto slab : regular_hyper_slabs) {
hyperslab |= slab;
}
```
This eventually triggers calling:
```
for(auto slab : regular_hyper_slabs) {
auto [offset, stride, counts, blocks] = slab;
H5Sselect_hyperslab(space_id, offset, stride, counts, block);
}
```
Measurements show that this has runtime that's quadratic in the number
of regular hyper slabs. This starts becoming prohibitive at 10k - 40k
slabs.
We noticed that `H5Scombine_select` does not suffer from the same
performance issue. This allows us to optimize (long) chain of `Op::Or`
using divide and conquer.
The current implementation only optimizes streaks of `Op::Or`.
a7b22df to
47b2bf2
Compare
jorblancoa
approved these changes
Jul 26, 2024
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A common pattern for creating semi-unstructured selection is to use many
(small) RegularHyperSlab and chain them:
This eventually triggers calling:
Measurements show that this has runtime that's quadratic in the number
of regular hyper slabs. This starts becoming prohibitive at 10k - 40k
slabs.
We noticed that
H5Scombine_selectdoes not suffer from the sameperformance issue. This allows us to optimize (long) chain of
Op::Orusing divide and conquer.
The current implementation only optimizes streaks of
Op::Or.