Skip to content

tikv_alloc: Add per thread memory usage (#16255)#18563

Merged
ti-chi-bot[bot] merged 1 commit intotikv:release-7.5from
ti-chi-bot:cherry-pick-16255-to-release-7.5
Jul 11, 2025
Merged

tikv_alloc: Add per thread memory usage (#16255)#18563
ti-chi-bot[bot] merged 1 commit intotikv:release-7.5from
ti-chi-bot:cherry-pick-16255-to-release-7.5

Conversation

@ti-chi-bot
Copy link
Member

@ti-chi-bot ti-chi-bot commented Jun 18, 2025

This is an automated cherry-pick of #16255

What is changed and how it works?

Issue Number: Ref #15927

What's Changed:

Add per thread memory usage

Jemalloc has mapped memory stats per arena. We can bind each thread with a dedicated arena, then we have per thread memory usage.

The total memory ~== block cache size + total of per thread footprint
截屏2024-04-07 16 28 25

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Release note

Add per thread memory usage

@ti-chi-bot ti-chi-bot added affects-7.5 This bug affects the 7.5.x(LTS) versions. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. type/cherry-pick-for-release-7.5 This PR is cherry-picked to release-7.5 from a source PR. labels Jun 18, 2025
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 18, 2025
Merged
9 tasks
@ti-chi-bot ti-chi-bot bot added the dco-signoff: yes Indicates the PR's author has signed the dco. label Jun 18, 2025
@ti-chi-bot
Copy link
Member Author

ti-chi-bot commented Jun 18, 2025

@hhwyt This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

@Connor1996 @hhwyt
tikv_alloc: Add per thread memory usage (tikv#16255)
c8ccfd6 
Signed-off-by: hhwyt <hhwyt1@gmail.com>
@hhwyt hhwyt force-pushed the cherry-pick-16255-to-release-7.5 branch from 98c3d57 to c8ccfd6 Compare June 18, 2025 10:27
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 18, 2025
@hhwyt hhwyt requested review from glorv and overvenus June 18, 2025 10:29
@hhwyt
Copy link
Contributor

hhwyt commented Jun 19, 2025 •
edited
Loading

Tested on my machine:
use grafana v8.5 :
image
use grafana v7.5:
image
image

The differences is because grafana v7.5 uses bar and stacked chart to display the metrics.

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 19, 2025
@hhwyt hhwyt requested review from Connor1996 and cfzjywxk June 19, 2025 03:41
@hhwyt
Copy link
Contributor

hhwyt commented Jun 19, 2025

@cfzjywxk PTAL, thx~

@hhwyt hhwyt mentioned this pull request Jun 19, 2025
Merged
9 tasks
@hhwyt
Copy link
Contributor

hhwyt commented Jun 19, 2025

Please also review the PR: #18564.

I will merge this one first, then that one.

Connor1996
Connor1996 approved these changes Jun 19, 2025
View reviewed changes
Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 19, 2025
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 19, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-06-19 03:40:16.050215018 +0000 UTC m=+329468.773393995: ☑️ agreed by glorv.
  • 2025-06-19 08:22:39.958987807 +0000 UTC m=+346412.682166789: ☑️ agreed by Connor1996.

@ti-chi-bot ti-chi-bot bot added the cherry-pick-approved Cherry pick PR approved by release team. label Jul 10, 2025
@ti-chi-bot ti-chi-bot bot added cherry-pick-approved Cherry pick PR approved by release team. do-not-merge/cherry-pick-not-approved and removed do-not-merge/cherry-pick-not-approved cherry-pick-approved Cherry pick PR approved by release team. labels Jul 10, 2025
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jul 11, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, Connor1996, glorv

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Jul 11, 2025
@hhwyt
Copy link
Contributor

hhwyt commented Jul 11, 2025

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 11, 2025
@hhwyt
Copy link
Contributor

hhwyt commented Jul 11, 2025

/retest

@hhwyt
Copy link
Contributor

hhwyt commented Jul 11, 2025

/merge

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jul 11, 2025

@hhwyt: We have migrated to builtin LGTM and approve plugins for reviewing.

👉 Please use /approve when you want to approve this pull request.

The changes announcement: LGTM plugin changes

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot bot merged commit 5ad8129 into tikv:release-7.5 Jul 11, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cfzjywxk cfzjywxk cfzjywxk approved these changes

@glorv glorv glorv approved these changes

@Connor1996 Connor1996 Connor1996 approved these changes

@overvenus overvenus Awaiting requested review from overvenus

Assignees

@hhwyt hhwyt

Labels

affects-7.5 This bug affects the 7.5.x(LTS) versions. approved cherry-pick-approved Cherry pick PR approved by release team. dco-signoff: yes Indicates the PR's author has signed the dco. lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. severity/minor size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. type/cherry-pick-for-release-7.5 This PR is cherry-picked to release-7.5 from a source PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ti-chi-bot @hhwyt @cfzjywxk @glorv @Connor1996