[FLASH-575] Improve the performance of stream operations in DM#296
Merged
flowbehappy merged 19 commits intoDeltaMergeEnginefrom Oct 30, 2019
Merged
[FLASH-575] Improve the performance of stream operations in DM#296flowbehappy merged 19 commits intoDeltaMergeEnginefrom
flowbehappy merged 19 commits intoDeltaMergeEnginefrom
Conversation
Contributor
There was a problem hiding this comment.
Maybe we should use min rather than max? The signature is IStorage::read( ..., size_t max_block_size, ...)
Contributor
Author
There was a problem hiding this comment.
Normally expected_block_size = DEFAULT_BLOCK_SIZE (65536)。While STABLE_CHUNK_ROWS = 8192 。
a1f4966 to
4cdd5fa
Compare
Contributor
Author
|
/run-integration-tests |
5 similar comments
Contributor
|
/run-integration-tests |
Contributor
|
/run-integration-tests |
Contributor
Author
|
/run-integration-tests |
Contributor
Author
|
/run-integration-tests |
Contributor
|
/run-integration-tests |
abb41fc to
94167b4
Compare
Contributor
Author
|
/run-integration-tests |
1 similar comment
Contributor
Author
|
/run-integration-tests |
Contributor
Author
|
/build |
Contributor
Author
|
/rebuild |
Contributor
Author
|
/run-integration-tests |
Contributor
|
/rebuild |
Contributor
|
/run-integration-tests |
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Oct 30, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 1, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 1, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 5, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mange table xxx statuscommand, to show the metrics about DM table.