Fix background gc when rows covered by delete range is larger than stable#3657
Fix background gc when rows covered by delete range is larger than stable#3657ti-chi-bot merged 15 commits intopingcap:masterfrom
Conversation
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. DetailsReviewer can indicate their review by submitting an approval review. |
JaySon-Huang
left a comment
There was a problem hiding this comment.
LGTM
@flowbehappy PTAL
|
Which version it affects? |
Before 5.3.0, we don't have a mechanism to completely delete data after removing tiflash replicas. We should cherry-pick relative PRs to 5.x to address this issue. CC @lidezhu |
flowbehappy
left a comment
There was a problem hiding this comment.
LGTM with a minor comment.
|
/rebuild |
|
/run-all-tests |
|
Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/494/cobertura/ lines: 42.5% (47529 out of 111815) |
|
/merge |
|
@lidezhu: It seems you want to merge this PR, I will help you trigger all the tests: /run-all-tests You only need to trigger If you have any questions about the PR merge process, please refer to pr process. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
This pull request has been accepted and is ready to merge. DetailsCommit hash: 004c7be |
|
Coverage detail: https://ci-internal.pingcap.net/job/tics_ghpr_unit_test/510/cobertura/ lines: 42.5% (47533 out of 111816) |
|
In response to a cherrypick label: new pull request created: #3694. |
…able (#3657) (#3694) * Fix gc mechanism when rows covered by delete range is larger than stable rows * remove check to forbid gc on small table * make sure delete range is not empty before gc * small refactor * remove obsolete header file * add some comment about the gc trigger criteria * add more comments * avoid gc on empty tables and avoid gc work triggered too much at the same interval * add some comment about BackgroundProcessingPool::addTask behaviour * small improvement on comments * add more comments * use old log macro Co-authored-by: lidezhu <lidezhu@pingcap.com> Co-authored-by: lidezhu <47731263+lidezhu@users.noreply.github.com>
What problem does this PR solve?
Issue Number: close #3659
Problem Summary: If some segments was generated by logical split, the new segments will keep the delete range of the old segment. And by the old gc logic, the rows in stable covered by delete range is larger than the valid rows in the stable which will skip gc on these segments.
The original purpose of this check is to prevent small segments to merge delta. But after some detail investigation, this check is really unnecessary.
What is changed and how it works?
Check List
Tests
Release note