sst_importer: set fill cache to false when merge downloaded SST files by Leavrth · Pull Request #19229 · tikv/tikv

Leavrth · 2025-12-24T02:39:07Z

What is changed and how it works?

Issue Number: Close #19228

What's Changed:

set fill cache to false when merge downloaded SST files

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Release note

None

Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>

Copilot

Pull request overview

This PR addresses a memory usage issue during PITR (Point-in-Time Recovery) restore operations by setting fill_cache to false when creating iterators over downloaded SST files. This prevents the RocksDB block cache from being populated with data that is only read once during the merge operation, reducing memory consumption during restore.

Sets fill_cache to false for SST file iterators during merge operations
Reduces memory pressure during PITR restore by avoiding unnecessary cache population

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ti-chi-bot · 2025-12-24T07:37:12Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3pointer, YuJuncen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [YuJuncen]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-12-24T07:37:14Z

[LGTM Timeline notifier]

Timeline:

2025-12-24 02:39:40.366465103 +0000 UTC m=+2218325.180242665: ☑️ agreed by YuJuncen.
2025-12-24 07:37:13.352484167 +0000 UTC m=+2236178.166261738: ☑️ agreed by 3pointer.

Leavrth · 2025-12-24T09:51:47Z

/retest

Leavrth · 2025-12-29T07:13:13Z

/retest

ti-chi-bot · 2025-12-29T08:41:34Z

In response to a cherrypick label: new pull request created to branch release-8.5: #19246.

ti-chi-bot · 2025-12-29T08:46:43Z

@Leavrth: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-unit-test	`6949073`	link	unknown	`/test pull-unit-test`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

…tikv#19229) close tikv#19228 set fill cache to false when merge downloaded SST files Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>

* sst_importer: set fill cache to false when merge downloaded SST files (#19229) close #19228 set fill cache to false when merge downloaded SST files Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * sst_importer: ensure cleanup of temporary files on error (#19240) ref #17283, close #19239 sst_importer: enhance failpoint handling and ensure cleanup of temporary files on error - Ensure temporary files are cleaned up when a download fails in sst_importer - Add a new failpoint `download_files_ext_after_download` to inject errors right after batch downloads complete, improving test coverage of cleanup paths Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * Squashed commit of the following: commit 5a14b2b Author: Juncen Yu <yujuncen@pingcap.com> Date: Mon Dec 29 15:11:06 2025 +0800 make clippy happy Signed-off-by: Juncen Yu <yujuncen@pingcap.com> commit ace595e Merge: bd5bff3 147cacc Author: Juncen Yu <yujuncen@pingcap.com> Date: Fri Dec 26 17:51:36 2025 +0800 Merge branch 'master' of https://github.com/tikv/tikv into resolve-compact-conflict commit bd5bff3 Merge: fde14ce 301984c Author: Juncen Yu <yujuncen@pingcap.com> Date: Mon Dec 22 12:10:59 2025 +0800 Merge branch 'master' of https://github.com/tikv/tikv into resolve-compact-conflict Signed-off-by: Juncen Yu <yujuncen@pingcap.com> commit fde14ce Author: Juncen Yu <yujuncen@pingcap.com> Date: Tue Mar 18 15:44:06 2025 +0800 added more checks Signed-off-by: Juncen Yu <yujuncen@pingcap.com> commit d0801cb Author: Juncen Yu <yujuncen@pingcap.com> Date: Tue Mar 18 15:27:14 2025 +0800 added more case about collapsed rollback Signed-off-by: Juncen Yu <yujuncen@pingcap.com> commit 6d2b994 Author: Juncen Yu <yujuncen@pingcap.com> Date: Tue Mar 18 14:52:40 2025 +0800 compact-log-backup: try to resolve conflict encountering during compacting Signed-off-by: Juncen Yu <yujuncen@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * *: upgrade cargo-deny version to make it compatible to further developments. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com> Co-authored-by: lucasliang <nkcs_lykx@hotmail.com>

…#19229) (#19246) close #19228 set fill cache to false when merge downloaded SST files Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Co-authored-by: Jianjun Liao <jianjun.liao@outlook.com>

set fill cache to false when merge downloaded sst

6949073

Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>

Copilot AI review requested due to automatic review settings December 24, 2025 02:39

ti-chi-bot bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 24, 2025

Copilot started reviewing on behalf of Leavrth December 24, 2025 02:39 View session

YuJuncen approved these changes Dec 24, 2025

View reviewed changes

ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved and removed do-not-merge/needs-triage-completed labels Dec 24, 2025

Copilot AI reviewed Dec 24, 2025

View reviewed changes

3pointer approved these changes Dec 24, 2025

View reviewed changes

ti-chi-bot bot added the lgtm label Dec 24, 2025

ti-chi-bot bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 24, 2025

ti-chi-bot bot merged commit 0c1fe2a into tikv:master Dec 29, 2025
13 of 19 checks passed

ti-chi-bot bot added this to the Pool milestone Dec 29, 2025

Leavrth added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Dec 29, 2025

ti-chi-bot mentioned this pull request Dec 29, 2025

sst_importer: set fill cache to false when merge downloaded SST files (#19229) #19246

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sst_importer: set fill cache to false when merge downloaded SST files#19229

sst_importer: set fill cache to false when merge downloaded SST files#19229
ti-chi-bot[bot] merged 1 commit intotikv:masterfrom
Leavrth:fill_cache

Leavrth commented Dec 24, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

ti-chi-bot bot commented Dec 24, 2025

Uh oh!

ti-chi-bot bot commented Dec 24, 2025

Uh oh!

Leavrth commented Dec 24, 2025

Uh oh!

Leavrth commented Dec 29, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Dec 29, 2025

Uh oh!

ti-chi-bot bot commented Dec 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

Leavrth commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is changed and how it works?

Related changes

Check List

Release note

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

ti-chi-bot bot commented Dec 24, 2025

Uh oh!

ti-chi-bot bot commented Dec 24, 2025

[LGTM Timeline notifier]

Uh oh!

Leavrth commented Dec 24, 2025

Uh oh!

Leavrth commented Dec 29, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Dec 29, 2025

Uh oh!

ti-chi-bot bot commented Dec 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Leavrth commented Dec 24, 2025 •

edited

Loading