db: add Iterator.RangeKeyChanged#1855
Merged
jbowens merged 1 commit intocockroachdb:masterfrom Aug 9, 2022
Merged
Conversation
Member
|
This change is |
7a6be0a to
5411897
Compare
5411897 to
a3c7286
Compare
erikgrinaker
approved these changes
Aug 4, 2022
Contributor
erikgrinaker
left a comment
There was a problem hiding this comment.
API and tests look good, eager to try this out!
nicktrav
approved these changes
Aug 8, 2022
Contributor
nicktrav
left a comment
There was a problem hiding this comment.
Nice! Nothing major. I also audited the tests fairly closely.
Perhaps we can cut a ticket for your TODO in the metamorphic tests? It would be good to have coverage there.
Reviewed 11 of 11 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jbowens)
iterator.go line 1004 at r1 (raw file):
key = upperBound } if i.rangeKey != nil {
Super nit: if this block is able to be inlined, could it be a function? I counted 7 instances that looked the same.
Add a new Iterator method, RangeKeyChanged() that indicates whether the state of range keys changed in the most recent iterator positioning operation. If this new method returns false, the user is guaranteed that the previously-returned range key buffers returned from RangeBounds and RangeKeys are still valid. Additionally, use the SpanChanged hook to flip a flag on the range iterator range key state, indicating that a new span was encountered. This allows the iterator to avoid copying the range key state if the current range key has not changed. This commit removes setRangeKey and always copies range key buffers to Iterator-owned buffers in order to provide the guarantee that the buffers returned by RangeKeyBounds and RangeKeys will be stable until the iterator's range key changes. Most of this commit's direct performance gains are during reverse iteration, which no longer must copy the range key on every step. It's expected that this change will allow additional performance optimizations higher up and in both directions, because users will no longer need to copy and compare range keys in order to determine if they changed. ``` name old time/op new time/op delta Iterator_RangeKeyMasking/range-keys-suffixes=@10/forward-10 2.37ms ±11% 2.30ms ±11% ~ (p=0.142 n=20+20) Iterator_RangeKeyMasking/range-keys-suffixes=@10/backward-10 3.28ms ± 4% 3.11ms ± 8% -5.17% (p=0.000 n=17+20) Iterator_RangeKeyMasking/range-keys-suffixes=@50/forward-10 2.10ms ± 8% 2.03ms ± 8% -3.29% (p=0.001 n=20+18) Iterator_RangeKeyMasking/range-keys-suffixes=@50/backward-10 2.78ms ± 3% 2.64ms ± 3% -5.22% (p=0.000 n=18+19) Iterator_RangeKeyMasking/range-keys-suffixes=@75/forward-10 963µs ± 1% 951µs ± 1% -1.32% (p=0.000 n=17+20) Iterator_RangeKeyMasking/range-keys-suffixes=@75/backward-10 1.43ms ±16% 1.32ms ± 5% -7.72% (p=0.000 n=19+20) Iterator_RangeKeyMasking/range-keys-suffixes=@100/forward-10 151µs ± 1% 149µs ± 1% -0.95% (p=0.000 n=19+19) Iterator_RangeKeyMasking/range-keys-suffixes=@100/backward-10 301µs ± 1% 308µs ± 2% +2.11% (p=0.000 n=19+18) CombinedIteratorSeek/range-key=false/batch=false-10 2.83µs ± 1% 2.81µs ± 1% -0.90% (p=0.007 n=9+10) CombinedIteratorSeek/range-key=false/batch=true-10 4.85µs ± 1% 4.58µs ± 0% -5.60% (p=0.000 n=9+10) CombinedIteratorSeek/range-key=true/batch=false-10 6.43µs ± 0% 6.01µs ± 1% -6.49% (p=0.000 n=10+10) CombinedIteratorSeek/range-key=true/batch=true-10 8.47µs ± 0% 8.10µs ± 0% -4.44% (p=0.000 n=10+10) ``` Close cockroachdb#1775.
a3c7286 to
39dbb6f
Compare
jbowens
commented
Aug 9, 2022
Contributor
Author
jbowens
left a comment
There was a problem hiding this comment.
TFTRs,!
fwiw, only iterators configured with permanent block-property filter lack coverage in the metamorphic tests currently, but I will follow up with expanding its coverage.
Reviewable status: 5 of 11 files reviewed, 1 unresolved discussion (waiting on @nicktrav)
iterator.go line 1004 at r1 (raw file):
Previously, nicktrav (Nick Travers) wrote…
Super nit: if this block is able to be inlined, could it be a function? I counted 7 instances that looked the same.
unfortunately, it doesn't inline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a new Iterator method RangeKeyChanged() that indicates whether the state
of range keys changed in the most recent iterator positioning operation. If
this new method returns false, the user is guaranteed that the
previously-returned range key buffers returned from RangeBounds and RangeKeys
are still valid.
Additionally, use the SpanChanged hook to flip a flag on the range iterator
range key state, indicating that a new span was encountered. This allows the
iterator to avoid copying the range key state if the current range key has not
changed. This commit removes setRangeKey and always copies range key buffers to
Iterator-owned buffers in order to provide the guarantee that the buffers
returned by RangeKeyBounds and RangeKeys will be stable until the iterator's
range key changes.
Most of this commit's direct performance gains are during reverse iteration,
which no longer must copy the range key on every step. It's expected that this
change will allow additional performance optimizations higher up and in both
directions, because users will no longer need to copy and compare range keys in
order to determine if they changed.
Close #1775.