Skip to content

causal_ts: fix issue #12680#12697

Merged
ti-chi-bot merged 7 commits intotikv:masterfrom
pingyu:fix-12680
May 30, 2022
Merged

causal_ts: fix issue #12680#12697
ti-chi-bot merged 7 commits intotikv:masterfrom
pingyu:fix-12680

Conversation

@pingyu
Copy link
Contributor

@pingyu pingyu commented May 30, 2022

Signed-off-by: pingyu yuping@pingcap.com

What is changed and how it works?

Issue Number: Close #12680

What's Changed:

Fix issue #12680.

In the scenario of region merge, the target region would merge some entries from source region with larger timestamps, when leader of source region is in another store with larger TSO batch than the store of target region's leader.
So we need a flush after commit merge.
Also see analysis in #12680.

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
    No.
  • Need to cherry-pick to the release branch
    release-6.1

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • No.

Release note

Fix issue #12680.

pingyu added 5 commits May 29, 2022 04:33
Issue Number: tikv#12680

Signed-off-by: pingyu <yuping@pingcap.com>
Signed-off-by: pingyu <yuping@pingcap.com>
Signed-off-by: pingyu <yuping@pingcap.com>
Signed-off-by: pingyu <yuping@pingcap.com>
Signed-off-by: pingyu <yuping@pingcap.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 30, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • Connor1996
  • iosmanthus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 30, 2022
@pingyu
Copy link
Contributor Author

pingyu commented May 30, 2022

/cc @BusyJay @Connor1996 @haojinming

@ti-chi-bot ti-chi-bot requested review from BusyJay and Connor1996 May 30, 2022 07:48
@ti-chi-bot
Copy link
Member

@pingyu: GitHub didn't allow me to request PR reviews from the following users: haojinming.

Note that only tikv members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

/cc @BusyJay @Connor1996 @haojinming

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pingyu
Copy link
Contributor Author

pingyu commented May 30, 2022

/cc @BusyJay @Connor1996 @haojinming

PTAL, thanks~

Copy link
Contributor

@haojinming haojinming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@ti-chi-bot
Copy link
Member

@haojinming: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

Details

In response to this:

LGTM~

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Copy link
Contributor

@haojinming haojinming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@ti-chi-bot
Copy link
Member

@haojinming: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

Details

In response to this:

LGTM~

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 30, 2022
Signed-off-by: pingyu <yuping@pingcap.com>
Copy link
Member

@iosmanthus iosmanthus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 30, 2022
@iosmanthus
Copy link
Member

/merge

@ti-chi-bot
Copy link
Member

@iosmanthus: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

DetailsCommit hash: 394f255

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 30, 2022
@ti-chi-bot ti-chi-bot merged commit 8d1742c into tikv:master May 30, 2022
@ti-srebot
Copy link
Contributor

cherry pick to release-6.1 failed

fn flush_timestamp(&self, region: &Region, reason: &'static str) {
fail::fail_point!("causal_observer_flush_timestamp", |_| ());

if let Err(err) = self.causal_ts_provider.flush() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it will be blocked every time a new leader is elected or a region is merged?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. It seems to be a performance issue when API v2 is enabled.

The P99 duration of timestamp flush is 1.0 ~ 1.5ms in a Kingsoft cloud env.
image

I will think about how to optimize this. As it's not easy to reproduce the causality issues (#12498, #12680), it may also mean that it is unnecessary to flush timestamp for many circumstances.

How do you think ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the ts can be invalidated and flushed asynchronously, then the latency be best optimized.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the ts can be invalidated and flushed asynchronously, then the latency be best optimized.

OK, get it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RawKV API V2 timestamp causality violation

7 participants