store/tikv: change backoff type for missed tiflash peer. (#24577)#24600
store/tikv: change backoff type for missed tiflash peer. (#24577)#24600ti-chi-bot merged 4 commits intopingcap:release-5.0from
Conversation
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
|
@ti-srebot: This cherry pick PR is for a release branch and has not yet been approved by release team. To merge this cherry pick, it must first be approved ( AFTER it has been approved by collaborators, please ping the release team in a comment to request a cherry pick review. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/run-all-tests |
|
@hanfei1991 you're already a collaborator in bot's repo. |
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. DetailsReviewer can indicate their review by submitting an approval review. |
|
/merge |
|
This pull request has been accepted and is ready to merge. DetailsCommit hash: f69f0ee |
|
@ti-srebot: Your PR was out of date, I have automatically updated it for you. At the same time I will also trigger all tests for you: /run-all-tests If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
cherry-pick #24577 to release-5.0
You can switch your code base to this Pull Request by using git-extras:
# In tidb repo: git pr https://github.com/pingcap/tidb/pull/24600After apply modifications, you can push your change to this PR via:
What problem does this PR solve?
Problem Summary:
After stopping a TiFlash store, we expect the tiflash queries would successfully fallback to TiKV mode. At first it really works, but after a little while (maybe several minutes), the "GetTiFlashRpcContext" function is able to detect the stale TiFlash store and always returns nil RPCContext. In this case, backoff will retry repeatly and raise "region unavailable" error which cannot trigger fallback mechanism.
What is changed and how it works?
What's Changed:
When tiflash rpc context is nil, it's mostly attributed to the long failed store which cannot be easily recovered. Actually we'd better to add a new backoff type such as "TiFlash peer miss". But for briefness, we work it around by setting it "TiFlash rpc error", which has the same effect.
Related changes
Check List
Tests
Release note