Scatter: make peer scatter logic same with the leader by bufferflies · Pull Request #6965 · tikv/pd

bufferflies · 2023-08-16T12:50:08Z

What problem does this PR solve?

Issue Number: Close #6962

What is changed and how does it work?

In past, PD conside peer distribution in the different group influenece by using `TotalCountByStore` , but not include the leader distribution. The max used situation is partition table. 
After this pr, TIDB call scatter api will use same group not different. 

ref:
https://github.com/tikv/pd/pull/3422
https://github.com/pingcap/tidb/pull/46156

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Code changes

Side effects

Related changes

Release note

None.

Signed-off-by: bufferflies <1045931706@qq.com>

ti-chi-bot · 2023-08-16T12:50:09Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

CabinfeverB
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: bufferflies <1045931706@qq.com>

pkg/schedule/scatter/region_scatterer.go

Signed-off-by: bufferflies <1045931706@qq.com>

codecov · 2023-08-22T09:51:08Z

Codecov Report

Merging #6965 (b8e2f61) into master (9a574ed) will increase coverage by 0.13%.
Report is 4 commits behind head on master.
The diff coverage is 85.71%.

❗ Current head b8e2f61 differs from pull request most recent head 9783c53. Consider uploading reports for the commit 9783c53 to get more accurate results

@@            Coverage Diff             @@
##           master    #6965      +/-   ##
==========================================
+ Coverage   74.33%   74.47%   +0.13%     
==========================================
  Files         433      427       -6     
  Lines       46097    45339     -758     
==========================================
- Hits        34265    33764     -501     
+ Misses       8830     8616     -214     
+ Partials     3002     2959      -43

Flag	Coverage Δ
unittests	`74.47% <85.71%> (+0.13%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

pkg/schedule/scatter/region_scatterer_test.go

pkg/schedule/scatter/region_scatterer.go

CabinfeverB · 2023-08-28T09:09:49Z

/check-issue-triage-complete

pkg/schedule/scatter/region_scatterer.go

Signed-off-by: bufferflies <1045931706@qq.com>

CabinfeverB

Which branches do we need to cherry pick

bufferflies · 2023-09-01T01:30:28Z

Which branches do we need to cherry pick

will follow pingcap/tidb#46156.

bufferflies · 2023-09-01T01:34:09Z

/merge

ti-chi-bot · 2023-09-01T01:34:10Z

@bufferflies: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-09-01T01:34:12Z

This pull request has been accepted and is ready to merge.

Details

Commit hash: 9783c53

ti-chi-bot · 2023-09-01T01:34:25Z

@bufferflies: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-09-01T01:50:47Z

In response to a cherrypick label: new pull request created to branch release-6.1: #7026.

close tikv#6962 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2023-09-01T01:51:24Z

In response to a cherrypick label: new pull request created to branch release-6.5: #7027.

close tikv#6962 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2023-09-01T01:52:01Z

In response to a cherrypick label: new pull request created to branch release-7.1: #7028.

close tikv#6962 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

close #6962 In past, PD conside peer distribution in the different group influenece by using `TotalCountByStore` , but not include the leader distribution. The max used situation is partition table. After this pr, TIDB call scatter api will use same group not different. ref: #3422 pingcap/tidb#46156 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: bufferflies <1045931706@qq.com> Co-authored-by: buffer <1045931706@qq.com> Co-authored-by: bufferflies <1045931706@qq.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

CabinfeverB · 2023-09-19T08:35:42Z

pkg/schedule/scatter/region_scatterer.go

 	for _, store := range stores {
-		storeCount := context.selectedPeer.TotalCountByStore(store.GetID())
+		storeCount := context.selectedPeer.Get(store.GetID(), group)
+		if store.GetID() == peer.GetId() {


if store.GetID() == peer.GetStoreId() {

close #6962 In past, PD conside peer distribution in the different group influenece by using `TotalCountByStore` , but not include the leader distribution. The max used situation is partition table. After this pr, TIDB call scatter api will use same group not different. ref: #3422 pingcap/tidb#46156 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: bufferflies <1045931706@qq.com> Co-authored-by: buffer <1045931706@qq.com> Co-authored-by: bufferflies <1045931706@qq.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

close tikv#6962 In past, PD conside peer distribution in the different group influenece by using `TotalCountByStore` , but not include the leader distribution. The max used situation is partition table. After this pr, TIDB call scatter api will use same group not different. ref: tikv#3422 pingcap/tidb#46156 Signed-off-by: bufferflies <1045931706@qq.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>

bufferflies added 2 commits August 16, 2023 19:53

scatter

dea649a

Signed-off-by: bufferflies <1045931706@qq.com>

simply scatter

f8f38fd

Signed-off-by: bufferflies <1045931706@qq.com>

ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. labels Aug 16, 2023

ti-chi-bot bot requested review from disksing and nolouch August 16, 2023 12:50

ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 16, 2023

remove total count

e77d70a

Signed-off-by: bufferflies <1045931706@qq.com>

CabinfeverB self-requested a review August 16, 2023 13:00

rename

3540f71

Signed-off-by: bufferflies <1045931706@qq.com>

bufferflies force-pushed the scatter_bug branch from 6ff1e36 to 3540f71 Compare August 16, 2023 13:01

pass ut

0654696

Signed-off-by: bufferflies <1045931706@qq.com>

rleungx reviewed Aug 18, 2023

View reviewed changes

pkg/schedule/scatter/region_scatterer.go Show resolved Hide resolved

ti-chi-bot bot added do-not-merge/needs-triage-completed and removed do-not-merge/needs-linked-issue labels Aug 22, 2023

bufferflies added 2 commits August 22, 2023 17:37

addition

78b52bd

Signed-off-by: bufferflies <1045931706@qq.com>

Merge branch 'master' into scatter_bug

b8e2f61

lhy1024 reviewed Aug 22, 2023

View reviewed changes

pkg/schedule/scatter/region_scatterer_test.go Show resolved Hide resolved

pkg/schedule/scatter/region_scatterer.go Show resolved Hide resolved

ti-chi-bot bot removed the do-not-merge/needs-triage-completed label Aug 28, 2023

CabinfeverB reviewed Aug 30, 2023

View reviewed changes

pkg/schedule/scatter/region_scatterer.go Outdated Show resolved Hide resolved

pkg/schedule/scatter/region_scatterer.go Show resolved Hide resolved

rleungx reviewed Aug 30, 2023

View reviewed changes

pkg/schedule/scatter/region_scatterer.go Outdated Show resolved Hide resolved

rleungx reviewed Aug 30, 2023

View reviewed changes

pkg/schedule/scatter/region_scatterer.go Outdated Show resolved Hide resolved

address comment

8412f8c

Signed-off-by: bufferflies <1045931706@qq.com>

bufferflies requested review from CabinfeverB and rleungx August 30, 2023 09:40

Merge branch 'master' into scatter_bug

9783c53

CabinfeverB approved these changes Aug 31, 2023

View reviewed changes

bufferflies requested a review from lhy1024 August 31, 2023 07:34

rleungx approved these changes Aug 31, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 31, 2023

ti-chi-bot bot added needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. needs-cherry-pick-release-6.5 Should cherry pick this PR to release-6.5 branch. needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. labels Sep 1, 2023

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 1, 2023

Merge branch 'master' into scatter_bug

a067ab5

ti-chi-bot bot merged commit 72a13c0 into tikv:master Sep 1, 2023

ti-chi-bot mentioned this pull request Sep 1, 2023

Scatter: make peer scatter logic same with the leader (#6965) #7026

Merged

ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Sep 1, 2023

This is an automated cherry-pick of tikv#6965

26fc291

close tikv#6962 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Sep 1, 2023

This is an automated cherry-pick of tikv#6965

62f8135

close tikv#6962 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Sep 1, 2023

Scatter: make peer scatter logic same with the leader (#6965) #7027

Merged

ti-chi-bot mentioned this pull request Sep 1, 2023

Scatter: make peer scatter logic same with the leader (#6965) #7028

Merged

ti-chi-bot pushed a commit to ti-chi-bot/pd that referenced this pull request Sep 1, 2023

This is an automated cherry-pick of tikv#6965

62057c0

close tikv#6962 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

bufferflies deleted the scatter_bug branch September 4, 2023 04:36

CabinfeverB reviewed Sep 19, 2023

View reviewed changes

Conversation

bufferflies commented Aug 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

Uh oh!

ti-chi-bot bot commented Aug 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

CabinfeverB commented Aug 28, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CabinfeverB left a comment

Choose a reason for hiding this comment

Uh oh!

bufferflies commented Sep 1, 2023

Uh oh!

bufferflies commented Sep 1, 2023

Uh oh!

ti-chi-bot bot commented Sep 1, 2023

Uh oh!

ti-chi-bot bot commented Sep 1, 2023

Uh oh!

ti-chi-bot bot commented Sep 1, 2023

Uh oh!

ti-chi-bot commented Sep 1, 2023

Uh oh!

ti-chi-bot commented Sep 1, 2023

Uh oh!

ti-chi-bot commented Sep 1, 2023

Uh oh!

CabinfeverB Sep 19, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bufferflies commented Aug 16, 2023 •

edited

Loading

ti-chi-bot bot commented Aug 16, 2023 •

edited

Loading

codecov bot commented Aug 22, 2023 •

edited

Loading