Skip to content

Rollup of ingest sst optimizations#18521

Merged
ti-chi-bot[bot] merged 7 commits intorelease-7.5-20250609-v7.5.6from
add-index-ingest-allow-write
Jun 10, 2025
Merged

Rollup of ingest sst optimizations#18521
ti-chi-bot[bot] merged 7 commits intorelease-7.5-20250609-v7.5.6from
add-index-ingest-allow-write

Conversation

@overvenus
Copy link
Member

@overvenus overvenus commented Jun 7, 2025 •
edited
Loading

What is changed and how it works?

Issue Number: Close #xxx

What's Changed:

Rollup of following sst ingestion optimizations

* raftstore: directly write kvs rather than ingestion when merging small regions. (#17408) (#18518)
* raftstore: remove stale ranges by DeleteByKeys rather than ingesting. (#18040) (#18519)
* raftstore: support rocksdb writes during ingestion #18096 (#18520)
* storage: support online config for flow-control module (#17396). (#18523)
* sst_importer: allow write during ingesting sst (#18514) (#18522)

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Release note

Optimize request tail latency during region balance and add index

LykxSassinator and others added 3 commits June 6, 2025 18:09
@LykxSassinator
raftstore: directly write kvs rather than ingestion when merging smal…
243dd52 
…l regions. (#17408) (#18518)

Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
@LykxSassinator
raftstore: remove stale ranges by DeleteByKeys rather than ingesting. (…
6a80753 
…#18040) (#18519)

Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
@LykxSassinator @overvenus
[hotifx-to-7.5] raftstore: support rocksdb writes during ingestion #1…
7d3f332 
…8096 (#18520)

* [hotifx-to-7.5] raftstore: support rocksdb writes during ingestion #18096

Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>

* Update rocksdb deps.

Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>

* Fix compile

Signed-off-by: Neil Shen <overvenus@gmail.com>

---------

Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
Signed-off-by: Neil Shen <overvenus@gmail.com>
Co-authored-by: Neil Shen <overvenus@gmail.com>
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 7, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/cherry-pick-not-approved dco-signoff: yes Indicates the PR's author has signed the dco. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 7, 2025
@overvenus
Copy link
Member Author

overvenus commented Jun 7, 2025

/test

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 7, 2025

@overvenus: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

/test pull-unit-test

The following commands are available to trigger optional jobs:

/debug pull-integration-test

Use /test all to run the following jobs that were automatically triggered:

tikv/tikv/release-7.5/pull_unit_test
Details

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@overvenus
Copy link
Member Author

overvenus commented Jun 7, 2025

/test pull-unit-test

@overvenus overvenus changed the title Rollup of ingest sst optimizations [WIP] Rollup of ingest sst optimizations Jun 7, 2025
@overvenus overvenus marked this pull request as ready for review June 7, 2025 08:08
LykxSassinator and others added 2 commits June 8, 2025 14:02
@LykxSassinator
[hotfix-to-7.5] storage: support online config for flow-control module (
936c043 
#17396). (#18523)

Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
@overvenus
sst_importer: allow write during ingesting sst (#18514) (#18522)
571a8a4 
ref #18081

Signed-off-by: Neil Shen <overvenus@gmail.com>
@overvenus overvenus changed the base branch from release-7.5 to release-7.5-20250609-v7.5.6 June 9, 2025 14:51
@overvenus overvenus changed the title [WIP] Rollup of ingest sst optimizations Rollup of ingest sst optimizations Jun 10, 2025
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 10, 2025
@overvenus
Address clippy warnings
36eb3e8 
Signed-off-by: Neil Shen <overvenus@gmail.com>
Connor1996
Connor1996 approved these changes Jun 10, 2025
View reviewed changes
Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 10, 2025
@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 10, 2025
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 10, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-06-10 05:04:59.291153286 +0000 UTC m=+331477.519468549: ☑️ agreed by Connor1996.
  • 2025-06-10 05:23:34.266962372 +0000 UTC m=+332592.495277632: ☑️ agreed by glorv.

@LykxSassinator
Copy link
Contributor

LykxSassinator commented Jun 10, 2025

/retest

@overvenus
Set write_global_seqno false for release-7.5
84cd04d 
Signed-off-by: Neil Shen <overvenus@gmail.com>
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 10, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, Connor1996, glorv, LykxSassinator

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Jun 10, 2025
@ti-chi-bot ti-chi-bot bot merged commit 7852e09 into release-7.5-20250609-v7.5.6 Jun 10, 2025
4 checks passed
@ti-chi-bot ti-chi-bot bot deleted the add-index-ingest-allow-write branch June 10, 2025 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cfzjywxk cfzjywxk cfzjywxk approved these changes

@glorv glorv glorv approved these changes

@Connor1996 Connor1996 Connor1996 approved these changes

@LykxSassinator LykxSassinator LykxSassinator approved these changes

Assignees

No one assigned

Labels

approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@overvenus @LykxSassinator @cfzjywxk @glorv @Connor1996