Skip to content

import_sst: add force partition range api#1340

Merged
ti-chi-bot[bot] merged 7 commits intopingcap:masterfrom
glorv:force-partition
Aug 27, 2025
Merged

import_sst: add force partition range api#1340
ti-chi-bot[bot] merged 7 commits intopingcap:masterfrom
glorv:force-partition

Conversation

@glorv
Copy link
Contributor

@glorv glorv commented Aug 20, 2025

Add 2 new function in the ImportSST service:

  • AddForcePartitionRange. Set a range in tikv that all incoming compact should partition at the region boundary as well as this range boundary to ensure new SST files under this range can be ingested to the bottom level. It will also try to do a manual compact(if needed) to partition potential existing huge SST files that overlaps with this range.
  • RemoveForcePartitionRange. Remove the range after the task(br/import/ddl) is finished.

@ti-chi-bot ti-chi-bot bot requested a review from terry1purcell August 20, 2025 07:22
@ti-chi-bot ti-chi-bot bot added the size/XXL label Aug 20, 2025
@glorv glorv changed the title add force partition range api import_sst: add force partition range api Aug 20, 2025
@glorv
Copy link
Contributor Author

glorv commented Aug 22, 2025

@D3Hunter @Connor1996 @3pointer PTAL thanks

@glorv
Copy link
Contributor Author

glorv commented Aug 22, 2025

/retest

Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

Cargo.toml Outdated
raft-proto = { git = "https://github.com/tikv/raft-rs", rev="95c532612ee6a83591fce9a8b51d6afe87b58835"}
protobuf-codegen = { git = "https://github.com/pingcap/rust-protobuf", rev="82b49fea7e696fd647b5aca0a6c6ec944eab3189" }
protobuf = { git = "https://github.com/pingcap/rust-protobuf", rev="82b49fea7e696fd647b5aca0a6c6ec944eab3189" }
protobuf-codegen = { git = "https://github.com/glorv/rust-protobuf", rev="e60d5bf8218328c98fc2aa44d1f0cf84a8b88578" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before merge, please use pingcap/rust-protobuf.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@glorv
Copy link
Contributor Author

glorv commented Aug 27, 2025

@D3Hunter @Connor1996 @BornChanger PTAL

Copy link
Contributor

@D3Hunter D3Hunter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm


message AddPartitionRangeRequest {
Range range = 1;
uint64 ttl = 2;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you comment the ttl unit?

@glorv
Copy link
Contributor Author

glorv commented Aug 27, 2025

/hold

Signed-off-by: glorv <glorvs@163.com>
@ti-chi-bot ti-chi-bot bot removed the lgtm label Aug 27, 2025
@glorv
Copy link
Contributor Author

glorv commented Aug 27, 2025

/unhold

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 27, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3pointer, D3Hunter, overvenus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 27, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-08-27 09:40:08.800053525 +0000 UTC m=+346700.677884478: ☑️ agreed by overvenus.
  • 2025-08-27 09:57:51.734739963 +0000 UTC m=+347763.612570917: ☑️ agreed by D3Hunter.
  • 2025-08-27 10:05:17.742386575 +0000 UTC m=+348209.620217529: ✖️🔁 reset by glorv.
  • 2025-08-27 10:06:32.710925407 +0000 UTC m=+348284.588756362: ☑️ agreed by D3Hunter.
  • 2025-08-27 10:16:54.20730443 +0000 UTC m=+348906.085135384: ☑️ agreed by 3pointer.

@ti-chi-bot ti-chi-bot bot merged commit 21e5378 into pingcap:master Aug 27, 2025
5 checks passed
@glorv glorv deleted the force-partition branch August 27, 2025 10:25
glorv added a commit to glorv/kvproto that referenced this pull request Sep 8, 2025
* add force partition range api

* fix typo

* patch rust-protobuf

* update rust-protobuf

* rename and add comment

Signed-off-by: glorv <glorvs@163.com>

* fix

---------

Signed-off-by: glorv <glorvs@163.com>
ti-chi-bot bot pushed a commit that referenced this pull request Sep 8, 2025
* add force partition range api

* fix typo

* patch rust-protobuf

* update rust-protobuf

* rename and add comment



* fix

---------

Signed-off-by: glorv <glorvs@163.com>
Leavrth pushed a commit to Leavrth/kvproto that referenced this pull request Oct 21, 2025
* add force partition range api

* fix typo

* patch rust-protobuf

* update rust-protobuf

* rename and add comment

Signed-off-by: glorv <glorvs@163.com>

* fix

---------

Signed-off-by: glorv <glorvs@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants