raftstore: remove stale ranges by DeleteByKeys rather than ingesting.#18040
raftstore: remove stale ranges by DeleteByKeys rather than ingesting.#18040ti-chi-bot[bot] merged 18 commits intotikv:masterfrom
Conversation
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
|
Skipping CI for Draft Pull Request. |
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
|
/retest |
|
/cc @hhwyt ptal |
|
@LykxSassinator: GitHub didn't allow me to request PR reviews from the following users: hhwyt, ptal. Note that only tikv members and repo collaborators can review this PR, and authors cannot review their own PRs. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
| ) | ||
| } else { | ||
| ( | ||
| DeleteStrategy::DeleteByWriter { |
There was a problem hiding this comment.
Looks like it becomes dead code after this PR, could you remove it?
There was a problem hiding this comment.
Maybe we can add additional annotations here for reviewers and developers, as this approach could be applied to other delete operations if necessary.
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
| } | ||
|
|
||
| fn delete_all_in_range(&self, ranges: &[Range<'_>]) -> Result<()> { | ||
| fn delete_all_in_range(&self, ranges: &[Range<'_>], forcely_by_key: bool) -> Result<()> { |
There was a problem hiding this comment.
Seems all places pass forcely_by_key = true
There was a problem hiding this comment.
Yep, introducing this extra parameter serves as a reminder to retain the DeleteByWriter strategy, which might be applied to other delete operations if necessary.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Connor1996, overvenus The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
ref tikv#18042, close tikv#18107 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
In response to a cherrypick label: new pull request created to branch |
…tikv#18040) Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
Rollup of following sst ingestion optimizations * raftstore: directly write kvs rather than ingestion when merging small regions. (#17408) (#18518) * raftstore: remove stale ranges by DeleteByKeys rather than ingesting. (#18040) (#18519) * raftstore: support rocksdb writes during ingestion #18096 (#18520) * storage: support online config for flow-control module (#17396). (#18523) * sst_importer: allow write during ingesting sst (#18514) (#18522) Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Signed-off-by: Neil Shen <overvenus@gmail.com> Co-authored-by: lucasliang <nkcs_lykx@hotmail.com>
|
/cherry-pick 8.1 |
|
@hhwyt: cannot checkout DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
/cherry-pick release-8.1 |
ref tikv#18042, close tikv#18107 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
@hhwyt: new pull request created to branch DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
ref tikv#18042, close tikv#18107 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
In response to a cherrypick label: new pull request created to branch |
…#18040) (#18622) ref #18042, close #18107 Optimizing the processing of clearing stale-ranges by DeleteByKeys, rather than ingesting. This optimization is used to mitigate the impacts, introduced by unnecessary ingesting sst files on latency when scaling. Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: lucasliang <nkcs_lykx@hotmail.com>
…#18040) (#18461) ref #18042, close #18107 Optimizing the processing of clearing stale-ranges by DeleteByKeys, rather than ingesting. This optimization is used to mitigate the impacts, introduced by unnecessary ingesting sst files on latency when scaling. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: lucasliang <nkcs_lykx@hotmail.com>
…#18040) (#18569) ref #18042, close #18107 Optimizing the processing of clearing stale-ranges by DeleteByKeys, rather than ingesting. This optimization is used to mitigate the impacts, introduced by unnecessary ingesting sst files on latency when scaling. Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: lucasliang <nkcs_lykx@hotmail.com>
What is changed and how it works?
Issue Number: Close #18107, Ref #18042
This PR mainly contains the following parts for optimization on scaling, used to mitigate the impacts, introduced by unnecessary ingesting sst files:
What's Changed:
Related changes
pingcap/docs/pingcap/docs-cn:Check List
Tests
Taking the
grpc messages durationin TiKV metrics panel as examples, positive performance feedbacks on reducing the long-tail latency can be reviewed from the following comparison results:grpc messages durationMeanwhile, the following E2E long-tail reduction also proves that this PR makes positive improvements effects:
Side effects
Release note