Doc: Add a design doc for optimizing gc safepoint advance#32934
Doc: Add a design doc for optimizing gc safepoint advance#32934ti-chi-bot merged 22 commits intopingcap:masterfrom
Conversation
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. DetailsReviewer can indicate their review by submitting an approval review. |
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
…into dev-design-docs
1 similar comment
|
@TonsnakeLin: GitHub didn't allow me to request PR reviews from the following users: reviewer. Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@TonsnakeLin: GitHub didn't allow me to request PR reviews from the following users: reviewer. Note that only pingcap members and repo collaborators can review this PR, and authors cannot review their own PRs. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
|
Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/a203649a9b7dac0f749b738debfa6b4c8eedce30 |
|
/run-unit-test |
|
/run-check_dev |
|
|
||
| ## Introduction | ||
|
|
||
| This document describes the design of the feature "optimize gc advance for internal transaction", which adds the startTS(start time stamp) of all internal transactions to the gc(garbage collect) safe point calculation process. |
There was a problem hiding this comment.
I see some "time stamp" and "safe point" usage, and I prefer "timestamp" and "safepoint".
|
|
||
| ## Motivation or Background | ||
|
|
||
| TiDB advances gc safe point every `tidb_gc_run_interval` time with step `tidb_gc_life_time`. If there is a long time transaction from user client lives more than `tidb_gc_life_time`, the safe point can't be advanced until the transaction is finished or lives over 24 hours. This mechanism ensures the continuous advancement of gc safe point and ensures that the data active transactions need to access will not be cleared. |
There was a problem hiding this comment.
TiDB advances gc safe point every
tidb_gc_run_intervaltime with steptidb_gc_life_time.
This statement is not accurate. Considering a cluster where nothing blocks safepoint's advancing, it's updated every tidb_gc_run_interval and the value are all now - tidb_gc_life_time, thus the step the safepoint advances each time is actually tidb_gc_run_interval too. Consider express it like:
TiDB advances gc safe point every
tidb_gc_run_intervaltime with the valuenow - tidb_gc_life_time.
|
|
||
| #### Store And Delete Internal Sessions | ||
|
|
||
| TiDB gets an internal session from a session pool implemented in `(s *session) getInternalSession`, when the transaction finished,it puts the session to the pool. This design stores the internal session to session manger and deletes the internal session from session manager in the function `(s *session) getInternalSession`. It calls the function `infosync.DeleteInternalSession()` to delete the internal session from session manager and calls the function `infosync.StoreInternalSession` to add the internal session to session manger. |
There was a problem hiding this comment.
| TiDB gets an internal session from a session pool implemented in `(s *session) getInternalSession`, when the transaction finished,it puts the session to the pool. This design stores the internal session to session manger and deletes the internal session from session manager in the function `(s *session) getInternalSession`. It calls the function `infosync.DeleteInternalSession()` to delete the internal session from session manager and calls the function `infosync.StoreInternalSession` to add the internal session to session manger. | |
| TiDB gets an internal session from a session pool implemented in `(s *session) getInternalSession`, when the transaction finished,it puts the session to the pool. This design stores the internal session to session manger and deletes the internal session from session manager in the function `(s *session) getInternalSession`. It calls the function `infosync.DeleteInternalSession()` to delete the internal session from session manager and calls the function `infosync.StoreInternalSession` to add the internal session to session manger. |
This design stores the internal session to session manger and deletes the internal session from session manager in the function
(s *session) getInternalSession.
It actually stores it in getInternalSession, but deletes it in the callback returned by getInternalSession. Consider making it more accurate.
|
|
||
| #### Calculate GC Safe Point | ||
|
|
||
| Currently, TiDB calculates gc safe point in the function `(is *InfoSyncer) ReportMinStartTS`. This design add some code in the function `ReportMinStartTS` to consider internal transactions when calculates gc safe point. |
There was a problem hiding this comment.
I'd prefer saying ReportMinStartTS calculates the minimum startTS of ongoing transactions (except the ones that exceed the limit), and it's result is useful for calculating the safe point, which is gc_worker's work. The ReportMinStartTS doesn't calculate the safe point directly.
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
…into dev-design-docs Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
|
/run-unit-test |
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
|
/merge |
|
This pull request has been accepted and is ready to merge. DetailsCommit hash: 9fcb652 |
TiDB MergeCI notify
|
What problem does this PR solve?
Issue Number: close #32725
Problem Summary:
What is changed and how it works?
Check List
Release note