Titan: Add a regression test for Titan disabling by v01dstar · Pull Request #18477 · tikv/tikv

v01dstar · 2025-05-23T02:01:49Z

What is changed and how it works?

Issue Number: Close #18434

What's Changed:

The real bug has been fixed by distinguishing Titan blob index from RocksDB blob index during RocksDB upgrade effort.
This PR just adds a regression test to prove it is fixed.
We reproduced the error with the same test code in 7.5

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Release note

None

ti-chi-bot · 2025-05-23T02:01:52Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Signed-off-by: Yang Zhang <yang.zhang@pingcap.com>

overvenus · 2025-05-30T06:46:03Z

tests/failpoints/cases/test_titan.rs

+    // blob db: file0 [k1: v]
+    assert!(!is_error_response(&block_on(
+        cluster
+            .async_add_peer(region1.get_id(), peer.clone())


Nit: It's better to initiate a transfer leader to the new peer after async_add_peer. Because it does not wait for the new peer to be created.

overvenus · 2025-05-30T06:52:11Z

Cargo.lock

 name = "raft-engine"
 version = "0.4.2"
-source = "git+https://github.com/tikv/raft-engine.git#de1ec937529e3a88e093db0cf0d403522565fe64"
+source = "git+https://github.com/tikv/raft-engine.git#392f5e66f8286dc1b6d7cf69f2bc20ed72d40123"


Upgrade by mistake?

ti-chi-bot · 2025-05-30T06:52:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LykxSassinator, overvenus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [LykxSassinator,overvenus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-05-30T06:52:28Z

[LGTM Timeline notifier]

Timeline:

2025-05-26 02:08:15.548082573 +0000 UTC m=+234825.919870033: ☑️ agreed by LykxSassinator.
2025-05-30 06:52:27.341009009 +0000 UTC m=+174568.403026628: ☑️ agreed by overvenus.

overvenus · 2025-05-30T08:20:38Z

tests/failpoints/cases/test_titan.rs

+    // Deletion record written by blob file GC is deleted by DeleteFiles, while the
+    // original record that is suppose to be deleted by DeleteByWriter is not
+    // deleted.
+    fail::cfg("after_delete_files_in_range", "return()").unwrap();


It seems to be an invalid fail point ...

rg after_delete_files_in_range tikv tikv/tests/failpoints/cases/test_titan.rs 125: fail::cfg("after_delete_files_in_range", "return()").unwrap();

hbisheng · 2025-05-30T08:21:51Z

tests/failpoints/cases/test_titan.rs

+        db.get_property_int_cf(defaultcf, "rocksdb.num-files-at-level5")
+            .unwrap()
+    );
+    // lv5: file1 [k1: deleted]


In this case, should k1 be rewritten into the LSM instead of getting deleted?

close tikv#18434 The real bug has been fixed by distinguishing Titan blob index from RocksDB blob index during RocksDB upgrade effort. This PR just adds a regression test to prove it is fixed. We reproduced the error with the same test code in 7.5 Signed-off-by: Yang Zhang <yang.zhang@pingcap.com> downgrade log level form error to warning downgrade log level from error to warning backup_stream: encode ts related field into meta file path (tikv#18482) close tikv#18497, ref pingcap/tidb#61318 backup_stream: encode ts related field into meta file path. Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: 3pointer <qdlc2010@gmail.com> Co-authored-by: 山岚 <36239017+YuJuncen@users.noreply.github.com>

close tikv#18497, ref pingcap/tidb#61318 backup_stream: encode ts related field into meta file path. Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: 3pointer <qdlc2010@gmail.com> Co-authored-by: 山岚 <36239017+YuJuncen@users.noreply.github.com> pd: degrade useless log (tikv#18504) close tikv#18506 Signed-off-by: bufferflies <1045931706@qq.com> tikv:downgrade log level from error to warning Titan: Add a regression test for Titan disabling (tikv#18477) close tikv#18434 The real bug has been fixed by distinguishing Titan blob index from RocksDB blob index during RocksDB upgrade effort. This PR just adds a regression test to prove it is fixed. We reproduced the error with the same test code in 7.5 Signed-off-by: Yang Zhang <yang.zhang@pingcap.com> backup_stream: encode ts related field into meta file path (tikv#18482) close tikv#18497, ref pingcap/tidb#61318 backup_stream: encode ts related field into meta file path. Signed-off-by: 3pointer <luancheng@pingcap.com> Signed-off-by: 3pointer <qdlc2010@gmail.com> Co-authored-by: 山岚 <36239017+YuJuncen@users.noreply.github.com> pd: degrade useless log (tikv#18504) close tikv#18506 Signed-off-by: bufferflies <1045931706@qq.com>

ti-chi-bot · 2025-06-13T06:16:23Z

In response to a cherrypick label: new pull request created to branch release-8.5: #18546.

close #18434 The real bug has been fixed by distinguishing Titan blob index from RocksDB blob index during RocksDB upgrade effort. This PR just adds a regression test to prove it is fixed. We reproduced the error with the same test code in 7.5 Signed-off-by: Yang Zhang <yang.zhang@pingcap.com> Co-authored-by: Yang Zhang <yang.zhang@pingcap.com>

ti-chi-bot bot added the do-not-merge/needs-linked-issue label May 23, 2025

v01dstar force-pushed the add-titan-test branch 2 times, most recently from fe02879 to 5b34be7 Compare May 23, 2025 07:22

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 23, 2025

Add a regression test for Titan disabling

1099b0d

Signed-off-by: Yang Zhang <yang.zhang@pingcap.com>

v01dstar force-pushed the add-titan-test branch from 5b34be7 to 1099b0d Compare May 23, 2025 07:23

v01dstar requested review from LykxSassinator and overvenus and removed request for LykxSassinator May 23, 2025 07:24

ti-chi-bot bot removed the do-not-merge/needs-linked-issue label May 23, 2025

v01dstar marked this pull request as ready for review May 23, 2025 07:26

v01dstar requested a review from LykxSassinator May 23, 2025 07:26

Fix clippy

8576265

Signed-off-by: Yang Zhang <yang.zhang@pingcap.com>

LykxSassinator approved these changes May 26, 2025

View reviewed changes

ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels May 26, 2025

overvenus approved these changes May 30, 2025

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels May 30, 2025

ti-chi-bot bot merged commit 41f160e into tikv:master May 30, 2025
8 checks passed

ti-chi-bot bot added this to the Pool milestone May 30, 2025

overvenus reviewed May 30, 2025

View reviewed changes

hbisheng reviewed May 30, 2025

View reviewed changes

v01dstar added affects-8.5 This bug affects the 8.5.x(LTS) versions. affects-9.0 This bug affects the 9.0.x versions. labels Jun 1, 2025

v01dstar deleted the add-titan-test branch June 13, 2025 06:15

v01dstar added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Jun 13, 2025

ti-chi-bot mentioned this pull request Jun 13, 2025

Titan: Add a regression test for Titan disabling (#18477) #18546

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Titan: Add a regression test for Titan disabling#18477

Titan: Add a regression test for Titan disabling#18477
ti-chi-bot[bot] merged 2 commits intotikv:masterfrom
v01dstar:add-titan-test

v01dstar commented May 23, 2025 •

edited

Loading

Uh oh!

ti-chi-bot bot commented May 23, 2025

Uh oh!

overvenus May 30, 2025

Uh oh!

overvenus May 30, 2025

Uh oh!

ti-chi-bot bot commented May 30, 2025

Uh oh!

ti-chi-bot bot commented May 30, 2025

Uh oh!

Uh oh!

overvenus May 30, 2025

Uh oh!

hbisheng May 30, 2025

Uh oh!

ti-chi-bot commented Jun 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

v01dstar commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is changed and how it works?

Related changes

Check List

Release note

Uh oh!

ti-chi-bot bot commented May 23, 2025

Uh oh!

overvenus May 30, 2025

Choose a reason for hiding this comment

Uh oh!

overvenus May 30, 2025

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot bot commented May 30, 2025

Uh oh!

ti-chi-bot bot commented May 30, 2025

[LGTM Timeline notifier]

Uh oh!

Uh oh!

overvenus May 30, 2025

Choose a reason for hiding this comment

Uh oh!

hbisheng May 30, 2025

Choose a reason for hiding this comment

Uh oh!

ti-chi-bot commented Jun 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

v01dstar commented May 23, 2025 •

edited

Loading