cdc: filter events with the observed range before load old values#17656
cdc: filter events with the observed range before load old values#17656ti-chi-bot[bot] merged 6 commits intotikv:release-7.5from
Conversation
Signed-off-by: qupeng <qupeng@pingcap.com>
| let resolver = resolver.as_mut(); | ||
| let entries = self.scan_batch(&mut scanner, cursors, resolver).await?; | ||
| let entries = self | ||
| .scan_batch(&mut scanner, cursors, resolver, &mut stats) |
There was a problem hiding this comment.
How about add start and end boundaries to the scanner?
There was a problem hiding this comment.
#17274 does such thing. But it's a huge refactor, we can't apply it to old versions. So here we just avoid unnecessary old-value loadings.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: 3AceShowHand, overvenus The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
|
/test |
|
@hicqu: The
The following commands are available to trigger optional jobs:
Use
DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/test pull-unit-test |
1 similar comment
|
/test pull-unit-test |
|
/retest |
1 similar comment
|
/retest |
|
/cherry-pick release-7.1 |
close tikv#16601, close tikv#17620 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
@hicqu: new pull request created to branch DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
close tikv#18142 cdc: fix the panic introduced by tikv#17656 Signed-off-by: qupeng <qupeng@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#18142 cdc: fix the panic introduced by tikv#17656 Signed-off-by: qupeng <qupeng@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#18142 cdc: fix the panic introduced by tikv#17656 Signed-off-by: qupeng <qupeng@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
|
/cherry-pick release-8.1 |
|
@3AceShowHand: new pull request created to branch DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
close tikv#16601, close tikv#17620 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
What is changed and how it works?
Issue Number: Close #16601
Issue Number: Close #17620
What's Changed:
Filter CDC events with the observed range before load old values. This can reduce CPU usage, especially for the situation with lots of continuous RocksDB Tombstones, which are generated from
drop/truncate table/partitionDDLs.Related changes
pingcap/docs/pingcap/docs-cn:Check List
Tests
Side effects
Release note