Skip to content

ignore delmark when add minmax for pk column (#4746)#4963

Merged
lidezhu merged 4 commits intopingcap:release-5.4-20220523from
lidezhu:fix-pk-minmax-index
May 23, 2022
Merged

ignore delmark when add minmax for pk column (#4746)#4963
lidezhu merged 4 commits intopingcap:release-5.4-20220523from
lidezhu:fix-pk-minmax-index

Conversation

@lidezhu
Copy link
Contributor

@lidezhu lidezhu commented May 23, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #4747

Problem Summary:
If rows in a stable pack is all mark deleted, then it's minmax index for pk column is both 0.
And when dt_enable_skippable_place is enabled and try to place delta index on this segment, it will filter out all pack which is not in the pk range [start_key, +infinite). So the previous stable pack is ignored.
But for the place delta index algorithm, it's expected that all the rows in the pk range [start_key, +infinite) is placed no matter whether it's deleted. So when we try to read this segment using the placed delta index, it reports error like "DeltaMerge return wrong result"

What is changed and how it works?

Ignore delmark when add minmax for pk column.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Fix potential query error when select on a table with many delete operations

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 23, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • JaySon-Huang
  • flowbehappy

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 23, 2022

@lidezhu: This cherry pick PR is for a release branch and has not yet been approved by release team.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick, it must first be approved by the collaborators.

AFTER it has been approved by collaborators, please ping the release team in a comment to request a cherry pick review.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot added do-not-merge/cherry-pick-not-approved release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels May 23, 2022
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 23, 2022
JaySon-Huang
JaySon-Huang approved these changes May 23, 2022
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 23, 2022
@lidezhu
Copy link
Contributor Author

lidezhu commented May 23, 2022

/run-all-tests

@lidezhu
Copy link
Contributor Author

lidezhu commented May 23, 2022

/run-unit-test

@sre-bot
Copy link
Collaborator

sre-bot commented May 23, 2022

lines: 43.3% (49829 out of 115138)
branches: 6.4% (81458 out of 1263981)

@flowbehappy
Copy link
Contributor

flowbehappy commented May 23, 2022

/merge

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 23, 2022

@flowbehappy: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 23, 2022

This pull request has been accepted and is ready to merge.

DetailsCommit hash: 7d7a08c

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 23, 2022
@flowbehappy
Copy link
Contributor

flowbehappy commented May 23, 2022

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented May 23, 2022

lines: 43.3% (49829 out of 115138)
branches: 6.4% (81461 out of 1263981)

@lidezhu
Copy link
Contributor Author

lidezhu commented May 23, 2022

1Euk0POcl6
Contacting with qa to fix the miss image.

@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label May 23, 2022
@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 23, 2022

Merge canceled because a new commit is pushed.

@ti-chi-bot ti-chi-bot removed the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 23, 2022
@ti-chi-bot ti-chi-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 23, 2022
@lidezhu lidezhu force-pushed the fix-pk-minmax-index branch from 95395e1 to ea8ee9a Compare May 23, 2022 12:38
@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 23, 2022
@lidezhu
Copy link
Contributor Author

lidezhu commented May 23, 2022

/run-all-tests

@lidezhu
Copy link
Contributor Author

lidezhu commented May 23, 2022

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented May 23, 2022

lines: 43.3% (49832 out of 115138)
branches: 6.4% (81471 out of 1263981)

@lidezhu lidezhu merged commit 87d4720 into pingcap:release-5.4-20220523 May 23, 2022
@lidezhu lidezhu deleted the fix-pk-minmax-index branch May 23, 2022 13:26
lidezhu added a commit to lidezhu/tics that referenced this pull request May 31, 2022
@lidezhu
ignore delmark when add minmax for pk column (pingcap#4746) (pingcap#…
03b2bd4 
…4963)

* ignore delmark when add minmax for pk column (pingcap#4746)

close pingcap#4747

* fix static analysis

* use release-5.4 branch image for other components

* fix fullstack image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@flowbehappy flowbehappy flowbehappy approved these changes

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

Assignees

No one assigned

Labels

do-not-merge/cherry-pick-not-approved release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@lidezhu @ti-chi-bot @sre-bot @flowbehappy @JaySon-Huang