*: Truncate partition with global index improvement#55831
*: Truncate partition with global index improvement#55831ti-chi-bot[bot] merged 95 commits intopingcap:masterfrom
Conversation
|
Hi @mjonss. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #55831 +/- ##
================================================
+ Coverage 72.8328% 75.2357% +2.4028%
================================================
Files 1672 1721 +49
Lines 462993 476569 +13576
================================================
+ Hits 337211 358550 +21339
+ Misses 104974 95919 -9055
- Partials 20808 22100 +1292
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Defined2014
left a comment
There was a problem hiding this comment.
LGTM, please move some lines back into StateDeleteReorganization
[LGTM Timeline notifier]Timeline:
|
|
@mjonss: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/retest |
|
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Defined2014, tangenta, winoros The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
|
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/retest |
|
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/retest |
|
@mjonss: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
What problem does this PR solve?
Issue Number: close #55819
Problem Summary:
During TRUNCATE PARTITION writes was blocked to partitions that was being truncated, also during
StateDeleteReorganization, meaning the DDL would block writes to all truncated partitions for the full duration of the DDL, in case of a Global Index.What changed and how does it work?
Instead of simply blocking writes to partitions being truncated, we are hiding the new partitions from Global Indexes
StateWriteOnlyand old partitions fromStateDeleteOnlyso that they cannot be seen, and not cause any anomaly, where you can find data or not depending on access method, i.e. it may be found if using Global Index, but not for table scan.A side effect of this is that we are still blocking duplicate keys, if a write to a global unique index from a
StateDeleteOnlyclient that collides with a not-yet cleaned up Global Index entry, or a write to a global unique index fromStateWriteOnly(old partition, to be truncated) collides with a newly written entry inStateDeleteOnly(using the new emptied partition).To not generate these kind of duplicate errors for too long period, duplicate key errors are not generated during
StateDeleteReorganizationif the duplicate comes from an old truncated partition, instead that global unique index entry is overwritten.Also refactored/deduplicated code for cleaning up the global index entries from dropped/truncated partitions, so both DROP and TRUNCATE PARTITION uses the same code.
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.