Skip to content

Use an adaptive aggressive GC strategy on Delta (PageStorage) (#1552)#1553

Merged
JaySon-Huang merged 2 commits intopingcap:release-4.0from
ti-srebot:release-4.0-1769158c418a
Mar 15, 2021
Merged

Use an adaptive aggressive GC strategy on Delta (PageStorage) (#1552)#1553
JaySon-Huang merged 2 commits intopingcap:release-4.0from
ti-srebot:release-4.0-1769158c418a

Conversation

@ti-srebot
Copy link
Collaborator

@ti-srebot ti-srebot commented Mar 12, 2021 •
edited by JaySon-Huang
Loading

cherry-pick #1552 to release-4.0
You can switch your code base to this Pull Request by using git-extras:

# In tics repo:
git pr https://github.com/pingcap/tics/pull/1553

After apply modifications, you can push your change to this PR via:

git push git@github.com:ti-srebot/tics.git pr/1553:release-4.0-1769158c418a

What problem does this PR solve?

Issue Number: close #1550

Problem Summary: TiFlash OOM cause by too many legacy PageFile are not GC

What is changed and how it works?

  • Add some settings to make TiFlash can overwrite gc threshold of PageFile
  • Try to do more aggressive GC when there are many Legacy PageFiles
  • Add a build target page_ctl to do manual gc offline

Related changes

  • Need to cherry-pick to the release branch 4.0

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Side effects

  • N/A

Release note

  • Use an adaptive aggressive data GC strategy to avoid OOM

@JaySon-Huang @ti-srebot
cherry pick #1552 to release-4.0
a198ed4 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Mar 12, 2021
Merged
@ti-srebot ti-srebot added CHERRY-PICK cherry pick status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug. labels Mar 12, 2021
@ti-srebot ti-srebot requested a review from flowbehappy March 12, 2021 20:02
@ti-srebot ti-srebot added this to the v4.0.12 milestone Mar 12, 2021
@JaySon-Huang
Fix conflict
d0853b7 
Signed-off-by: JaySon-Huang <jayson.hjs@gmail.com>
@JaySon-Huang
Copy link
Contributor

JaySon-Huang commented Mar 13, 2021

/run-all-tests

2 similar comments
@JaySon-Huang
Copy link
Contributor

JaySon-Huang commented Mar 14, 2021

/run-all-tests

@solotzg
Copy link
Contributor

solotzg commented Mar 15, 2021

/run-all-tests

flowbehappy
flowbehappy approved these changes Mar 15, 2021
View reviewed changes
Copy link
Contributor

@flowbehappy flowbehappy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JaySon-Huang JaySon-Huang merged commit 5782d41 into pingcap:release-4.0 Mar 15, 2021
@JaySon-Huang JaySon-Huang deleted the release-4.0-1769158c418a branch March 15, 2021 07:40
LittleFall referenced this pull request in LittleFall/tics Mar 21, 2021
@ti-srebot @JaySon-Huang @LittleFall
Use an adaptive aggressive GC strategy on Delta (PageStorage) (#1552)…
081454c 
… (#1553)

* cherry pick #1552 to release-4.0
* Fix conflict

Co-authored-by: JaySon-Huang <jayson.hjs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@flowbehappy flowbehappy flowbehappy approved these changes

Assignees

@JaySon-Huang JaySon-Huang

Labels

CHERRY-PICK cherry pick status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug.

Projects

None yet

Milestone

v4.0.12

Development

Successfully merging this pull request may close these issues.

4 participants

@ti-srebot @JaySon-Huang @solotzg @flowbehappy