[DM] Support range read and lock optimization#140
Merged
flowbehappy merged 12 commits intoDeltaMergeEnginefrom Aug 5, 2019
Merged
[DM] Support range read and lock optimization#140flowbehappy merged 12 commits intoDeltaMergeEnginefrom
flowbehappy merged 12 commits intoDeltaMergeEnginefrom
Conversation
dbms/src/Storages/DeltaMerge/DMVersionFilterBlockInputStream.cpp
Outdated
Show resolved
Hide resolved
dbms/src/Storages/DeltaMerge/DMVersionFilterBlockInputStream.cpp
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Should we return at least one stream? If sorted_ranges is [(+∞, +∞)] (line 227), then we will return 0 stream
Contributor
Author
There was a problem hiding this comment.
If so, tasks.size() should be 1.
Contributor
There was a problem hiding this comment.
But here it will end with tasks.size() == 0 if we pass [(Int64::max, Int64::max)]. (Int64::max, Int64::max) intersect (Int64::min, Int64::max) return false in https://github.com/pingcap/tics/pull/140/files#diff-cc98b6f41bd336729fc39462f72a29d3R287
Contributor
There was a problem hiding this comment.
If user use UInt64 as pk, and pass range between [Int64::MAX~UInt64::MAX] ?
Contributor
Author
There was a problem hiding this comment.
Yes, we need to considerate this situation. But later https://internal.pingcap.net/jira/browse/FLASH-392
JaySon-Huang
reviewed
Aug 1, 2019
JaySon-Huang
reviewed
Aug 1, 2019
Contributor
Author
|
/run-integration-tests |
Contributor
Author
|
/build |
JaySon-Huang
reviewed
Aug 2, 2019
b9d8c38 to
5f16c2f
Compare
Contributor
Author
|
/run-integration-tests |
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Aug 26, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Sep 5, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Sep 12, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Sep 18, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Sep 23, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
that referenced
this pull request
Sep 26, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Oct 17, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Oct 18, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
that referenced
this pull request
Oct 22, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Oct 23, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Oct 30, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 1, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 1, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 5, 2019
* Segment support range scan * Bug fix: ColumnVector insertRangeFrom out of range * Only read tag column when necessary * print more info on sidLowerBound exception * Refactor DeltaMergeBlockInputStream * Optimize DMVersionFilterBlockInputStream, try unroll loops. * Support multi range read * bug fix * Optimize synchronization mechanism between read/write thread, decrease latency. * Fix testes * Address comments * fix typo
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note that there could be performance issues and bugs for this feature. So we currently by default disable this feature and will come back to it when implementing sparse index.