log backup: fix several issues during compact log backup. by 3pointer · Pull Request #18298 · tikv/tikv

3pointer · 2025-03-13T10:15:36Z

What is changed and how it works?

Issue Number: Close #18308

What's Changed:
1. GCS Atomic Write: Removed the content length=0 check to support atomic writes, aligning with S3 behavior.
2. Assertion Fix: Fixed assertion failures caused by unexpected rollback transactions. see details(#18300)
3. Duplicate File Fix: Resolved issues with duplicate output files causing compacted restore failures.
4. Enhanced logging to reduce noise.

log backup compact: fix several issues during compact a log backup

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Release note

None

ti-chi-bot · 2025-03-13T10:15:41Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

3pointer · 2025-03-17T09:44:15Z

/ok-to-test

Signed-off-by: 3pointer <luancheng@pingcap.com>

3pointer · 2025-03-18T03:28:46Z

/retest

Signed-off-by: 3pointer <luancheng@pingcap.com>

3pointer · 2025-03-18T03:34:33Z

/test pull-unit-test

Signed-off-by: 3pointer <luancheng@pingcap.com>

YuJuncen · 2025-03-18T07:17:45Z

components/compact-log-backup/src/test_util.rs

+            let file_name = sst_path
+                .file_name()
+                .expect("SST path should have a file name")
+                .to_str()


why not directly file.get_name()?

because file.get_name() is compaction_xxx/outputs/xxx_xx_x_x_xx.sst

YuJuncen

rest lgtm

YuJuncen · 2025-03-18T07:25:51Z

components/compact-log-backup/src/test_util.rs

+            assert_eq!(
+                parts.len(),
+                5,
+                "SST file name should have 5 parts separated by '_'"


Also show the full file name here.

YuJuncen · 2025-03-18T07:26:09Z

components/compact-log-backup/src/test_util.rs

+            let uuid_part = parts[4].trim_end_matches(".sst");
+            assert!(
+                uuid::Uuid::parse_str(uuid_part).is_ok(),
+                "Invalid UUID format in SST file name"


Also show the full sst name here.

ti-chi-bot · 2025-03-18T07:53:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Leavrth, YuJuncen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [YuJuncen]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-03-18T07:53:05Z

[LGTM Timeline notifier]

Timeline:

2025-03-18 07:26:50.989763027 +0000 UTC m=+340504.673999126: ☑️ agreed by YuJuncen.
2025-03-18 07:53:04.761538853 +0000 UTC m=+342078.445774949: ☑️ agreed by Leavrth.

ti-chi-bot · 2025-03-18T09:42:21Z

In response to a cherrypick label: new pull request created to branch release-9.0-beta.1: #18316.

…18316) close #18308 log backup compact: fix several issues during compact a log backup Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> Co-authored-by: 3pointer <luancheng@pingcap.com> Co-authored-by: Juncen Yu <yujuncen@pingcap.com>

close tikv#18308 log backup compact: fix several issues during compact a log backup Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com>

* br: batch download and merge download sst before ingest (#19062) close #19086 Add a new PRC method called batch-download to download batch SST. Signed-off-by: RidRisR <79858083+RidRisR@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * fix build Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * make format Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * implement the base library for compacting logs (#17632) close #17631 Added a new crate named `compact-log-backup`. Now it can merge some log files generated by log backup and make them become SSTs. Signed-off-by: hillium <yujuncen@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * added `compact-log-bakcup` to `tikv-ctl` (#17845) close #17844 Signed-off-by: hillium <yujuncen@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: record `min_input_ts` and `max_input_ts` in Compaction (#18085) close #18084 `min_input_ts` and `max_input_ts` will present in a log files compaction. Signed-off-by: hillium <yu745514916@live.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: fix typo (#18090) ref #15990 Fixed a typo: `Migartion` -> `Migration`. Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: filter out meta files by migration (#18123) close #18122 Now, `StreamMetaStorage` is able to filter out files by meta edits. Signed-off-by: hillium <yu745514916@live.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: added minimal compactions size (#18235) close #18234 Added `--minimal-compact-size` to `compact-log-backup`. Signed-off-by: hillium <yujuncen@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * log backup: fix several issues during compact log backup. (#18298) close #18308 log backup compact: fix several issues during compact a log backup Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: correct version assignment in subcompaction metadata (#18389) close #18390 Fixed a bug that caused the time range of compaction generated SSTs are too huge. Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: add new field to track fully compacted data KV files and fix metafile filtering (#18837) close #18843 compact_log_backup: add new field to track fully compacted data KV files and fix metafile filtering Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: use max ts among all storage checkpoint ts (#18848) close #18847 Now, `consistency` hook checks the storage checkpoint by the max value among them. Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: fix compact meta edit filter (#18842) close #18843 Merge the same meta edit from different migrations instead of replacing. Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: offload reading meta to diff cpus (#18885) close #18884 This PR spawns read s3 file tasks to remote threads. Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: read meta from checkpoint (#19068) close #19069 This PR makes `compact-log-backup` fills the migration with subcompactions skipped by checkpoint. Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * fix build Signed-off-by: Juncen Yu <yujuncen@pingcap.com> --------- Signed-off-by: RidRisR <79858083+RidRisR@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Signed-off-by: 山岚 <36239017+YuJuncen@users.noreply.github.com> Co-authored-by: ris <79858083+RidRisR@users.noreply.github.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Co-authored-by: 3pointer <luancheng@pingcap.com> Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com>

* br: batch download and merge download sst before ingest (tikv#19062) close tikv#19086 Add a new PRC method called batch-download to download batch SST. Signed-off-by: RidRisR <79858083+RidRisR@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * fix build Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * make format Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * implement the base library for compacting logs (tikv#17632) close tikv#17631 Added a new crate named `compact-log-backup`. Now it can merge some log files generated by log backup and make them become SSTs. Signed-off-by: hillium <yujuncen@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * added `compact-log-bakcup` to `tikv-ctl` (tikv#17845) close tikv#17844 Signed-off-by: hillium <yujuncen@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: record `min_input_ts` and `max_input_ts` in Compaction (tikv#18085) close tikv#18084 `min_input_ts` and `max_input_ts` will present in a log files compaction. Signed-off-by: hillium <yu745514916@live.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: fix typo (tikv#18090) ref tikv#15990 Fixed a typo: `Migartion` -> `Migration`. Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: filter out meta files by migration (tikv#18123) close tikv#18122 Now, `StreamMetaStorage` is able to filter out files by meta edits. Signed-off-by: hillium <yu745514916@live.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: added minimal compactions size (tikv#18235) close tikv#18234 Added `--minimal-compact-size` to `compact-log-backup`. Signed-off-by: hillium <yujuncen@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * log backup: fix several issues during compact log backup. (tikv#18298) close tikv#18308 log backup compact: fix several issues during compact a log backup Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: correct version assignment in subcompaction metadata (tikv#18389) close tikv#18390 Fixed a bug that caused the time range of compaction generated SSTs are too huge. Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: add new field to track fully compacted data KV files and fix metafile filtering (tikv#18837) close tikv#18843 compact_log_backup: add new field to track fully compacted data KV files and fix metafile filtering Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: use max ts among all storage checkpoint ts (tikv#18848) close tikv#18847 Now, `consistency` hook checks the storage checkpoint by the max value among them. Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: fix compact meta edit filter (tikv#18842) close tikv#18843 Merge the same meta edit from different migrations instead of replacing. Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: offload reading meta to diff cpus (tikv#18885) close tikv#18884 This PR spawns read s3 file tasks to remote threads. Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * compact_log_backup: read meta from checkpoint (tikv#19068) close tikv#19069 This PR makes `compact-log-backup` fills the migration with subcompactions skipped by checkpoint. Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> * fix build Signed-off-by: Juncen Yu <yujuncen@pingcap.com> --------- Signed-off-by: RidRisR <79858083+RidRisR@users.noreply.github.com> Signed-off-by: Juncen Yu <yujuncen@pingcap.com> Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Signed-off-by: 山岚 <36239017+YuJuncen@users.noreply.github.com> Co-authored-by: ris <79858083+RidRisR@users.noreply.github.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Co-authored-by: 3pointer <luancheng@pingcap.com> Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com>

3pointer changed the title ~~log backup compact: fix several issues during log backup.~~ [DNM] log backup compact: fix several issues during log backup. Mar 13, 2025

ti-chi-bot bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 13, 2025

ti-chi-bot bot added ok-to-test Indicates a PR is ready to be tested. do-not-merge/needs-triage-completed and removed do-not-merge/needs-linked-issue labels Mar 17, 2025

fix several issues and enhance logs

e03c521

Signed-off-by: 3pointer <luancheng@pingcap.com>

3pointer force-pushed the fix_compact_issues branch from 0bcc72d to e03c521 Compare March 17, 2025 09:54

ti-chi-bot bot removed the do-not-merge/needs-triage-completed label Mar 17, 2025

make clippy

9fb86da

Signed-off-by: 3pointer <luancheng@pingcap.com>

3pointer force-pushed the fix_compact_issues branch from 7d35a17 to 9fb86da Compare March 18, 2025 03:33

3pointer added 2 commits March 18, 2025 11:47

still enable assert to make process more safe

2cf77b3

Signed-off-by: 3pointer <luancheng@pingcap.com>

add test case for sst file name

ea89b06

Signed-off-by: 3pointer <luancheng@pingcap.com>

3pointer marked this pull request as ready for review March 18, 2025 04:03

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 18, 2025

3pointer changed the title ~~[DNM] log backup compact: fix several issues during log backup.~~ log backup: fix several issues during compact log backup. Mar 18, 2025

3pointer added 2 commits March 18, 2025 12:24

format code

c025d94

Signed-off-by: 3pointer <luancheng@pingcap.com>

Merge branch 'master' into fix_compact_issues

feb917f

YuJuncen reviewed Mar 18, 2025

View reviewed changes

YuJuncen approved these changes Mar 18, 2025

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Mar 18, 2025

ti-chi-bot bot added the approved label Mar 18, 2025

Leavrth approved these changes Mar 18, 2025

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Mar 18, 2025

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 18, 2025

ti-chi-bot bot merged commit 2191b16 into tikv:master Mar 18, 2025
8 checks passed

ti-chi-bot bot added this to the Pool milestone Mar 18, 2025

3pointer added the needs-cherry-pick-release-9.0-beta.1 Should cherry pick this PR to release-9.0-beta.1 branch. label Mar 18, 2025

ti-chi-bot mentioned this pull request Mar 18, 2025

log backup: fix several issues during compact log backup. (#18298) #18316

Merged

9 tasks

YuJuncen mentioned this pull request Dec 5, 2025

tikv-ctl: pick compact-log-backup related stuff to release-8.5 #19169

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

log backup: fix several issues during compact log backup. #18298

log backup: fix several issues during compact log backup. #18298
ti-chi-bot[bot] merged 6 commits intotikv:masterfrom
3pointer:fix_compact_issues

3pointer commented Mar 13, 2025 •

edited

Loading

Uh oh!

ti-chi-bot bot commented Mar 13, 2025

Uh oh!

3pointer commented Mar 17, 2025

Uh oh!

3pointer commented Mar 18, 2025

Uh oh!

3pointer commented Mar 18, 2025

Uh oh!

YuJuncen Mar 18, 2025 •

edited

Loading

Uh oh!

3pointer Mar 18, 2025 •

edited

Loading

Uh oh!

YuJuncen left a comment

Uh oh!

YuJuncen Mar 18, 2025

Uh oh!

YuJuncen Mar 18, 2025

Uh oh!

ti-chi-bot bot commented Mar 18, 2025

Uh oh!

ti-chi-bot bot commented Mar 18, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Mar 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

3pointer commented Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is changed and how it works?

Related changes

Check List

Release note

Uh oh!

ti-chi-bot bot commented Mar 13, 2025

Uh oh!

3pointer commented Mar 17, 2025

Uh oh!

3pointer commented Mar 18, 2025

Uh oh!

3pointer commented Mar 18, 2025

Uh oh!

YuJuncen Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

3pointer Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YuJuncen left a comment

Choose a reason for hiding this comment

Uh oh!

YuJuncen Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

YuJuncen Mar 18, 2025

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot bot commented Mar 18, 2025

Uh oh!

ti-chi-bot bot commented Mar 18, 2025

[LGTM Timeline notifier]

Uh oh!

Uh oh!

ti-chi-bot commented Mar 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

3pointer commented Mar 13, 2025 •

edited

Loading

YuJuncen Mar 18, 2025 •

edited

Loading

3pointer Mar 18, 2025 •

edited

Loading