Skip to content

config: adjust default value of config rocksdb.max-manifest-file-size from 128MiB to 256MiB#18496

Merged
ti-chi-bot[bot] merged 3 commits intotikv:masterfrom
glorv:manifest-size
Aug 28, 2025
Merged

config: adjust default value of config rocksdb.max-manifest-file-size from 128MiB to 256MiB#18496
ti-chi-bot[bot] merged 3 commits intotikv:masterfrom
glorv:manifest-size

Conversation

@glorv
Copy link
Contributor

@glorv glorv commented Jun 3, 2025

What is changed and how it works?

Issue Number: close #18996

What's Changed:

The default value of config `rocksdb.max-manifest-file-size` is too small for large clusters with more than 100k sst files. It can cause long ingest latency when the effective manifest file size exceed the threshold. So this PR adjust the default value to 256MiB which should be a large enough value.

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Release note

None

Signed-off-by: glorv <glorvs@163.com>
@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 3, 2025
Copy link
Member

@hbisheng hbisheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any downside to increasing the value? Start-up time may be longer due to the parsing of the manifest file?

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 3, 2025
Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 3, 2025
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 3, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-06-03 03:04:02.691087734 +0000 UTC m=+324528.893432999: ☑️ agreed by hbisheng.
  • 2025-06-03 10:06:03.908845781 +0000 UTC m=+349850.111191043: ☑️ agreed by Connor1996.

Copy link
Contributor

@LykxSassinator LykxSassinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to create a corresponding PR for docs-cn to document this change.

@ti-chi-bot ti-chi-bot bot added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Jul 4, 2025
@glorv
Copy link
Contributor Author

glorv commented Aug 6, 2025

/retest

@glorv glorv added needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. labels Aug 6, 2025
@glorv
Copy link
Contributor Author

glorv commented Aug 6, 2025

@zhangjinpeng87 PTAL, thanks

Signed-off-by: glorv <glorvs@163.com>
@glorv glorv changed the title config: adjust default value of config rocksdb.max-manifest-file-size from 128MiB to 512MiB config: adjust default value of config rocksdb.max-manifest-file-size from 128MiB to 256MiB Aug 6, 2025
Signed-off-by: glorv <glorvs@163.com>
@glorv
Copy link
Contributor Author

glorv commented Aug 11, 2025

@zhangjinpeng87 PTAL thanks

Copy link
Member

@zhangjinpeng87 zhangjinpeng87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Aug 28, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Connor1996, hbisheng, LykxSassinator, zhangjinpeng87

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Aug 28, 2025
@ti-chi-bot ti-chi-bot bot merged commit af47271 into tikv:master Aug 28, 2025
8 checks passed
@ti-chi-bot ti-chi-bot bot added this to the Pool milestone Aug 28, 2025
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #18887.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #18888.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #18889.

@ti-chi-bot ti-chi-bot bot removed the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Aug 29, 2025
yibin87 pushed a commit to yibin87/tikv that referenced this pull request Sep 2, 2025
…e` from 128MiB to 256MiB (tikv#18496)

ref tikv#15990

The default value of config `rocksdb.max-manifest-file-size` is too small for large clusters with more than 100k sst files. It can cause long ingest latency when the effective manifest file size exceed the threshold. So this PR adjust the default value to 256MiB which should be a large enough value.

Signed-off-by: glorv <glorvs@163.com>
@glorv glorv deleted the manifest-size branch September 23, 2025 06:49
ti-chi-bot bot pushed a commit that referenced this pull request Sep 24, 2025
…e` from 128MiB to 256MiB (#18496) (#18889)

close #18996

The default value of config `rocksdb.max-manifest-file-size` is too small for large clusters with more than 100k sst files. It can cause long ingest latency when the effective manifest file size exceed the threshold. So this PR adjust the default value to 256MiB which should be a large enough value.

Signed-off-by: glorv <glorvs@163.com>

Co-authored-by: glorv <glorvs@163.com>
3AceShowHand pushed a commit to 3AceShowHand/tikv that referenced this pull request Oct 13, 2025
…e` from 128MiB to 256MiB (tikv#18496)

ref tikv#15990

The default value of config `rocksdb.max-manifest-file-size` is too small for large clusters with more than 100k sst files. It can cause long ingest latency when the effective manifest file size exceed the threshold. So this PR adjust the default value to 256MiB which should be a large enough value.

Signed-off-by: glorv <glorvs@163.com>
Signed-off-by: 3AceShowHand <jinl1037@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved dco-signoff: yes Indicates the PR's author has signed the dco. lgtm needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

the default rocksdb manifest file size is too small

6 participants