Skip to content

Refactor DDL Module to support a large number of tables with frequent ddl actions#7437

Merged
ti-chi-bot[bot] merged 88 commits intopingcap:masterfrom
hongyunyan:hongyunyan_ddl_demo
Jun 15, 2023
Merged

Refactor DDL Module to support a large number of tables with frequent ddl actions#7437
ti-chi-bot[bot] merged 88 commits intopingcap:masterfrom
hongyunyan:hongyunyan_ddl_demo

Conversation

@hongyunyan
Copy link
Contributor

@hongyunyan hongyunyan commented May 8, 2023 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #7630 close #6532 #3777 #1664

Problem Summary:

Refactor DDL module to support a large number of tables with frequent ddl actions.

related PR:tikv/client-c#146

What is changed and how it works?

  1. new DDL architecture, basiclly including TiDB/Schema/xxx.
  2. change to use new ddl architecture in the main logical, mainly includes StorageDeltaMerge.cpp, DeltaMergeStore.cpp
  3. update and add a lot of tests for new DDL architecture

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Shorten TiFlash restart time, and be more efficient under multiple ddl

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented May 8, 2023

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels May 8, 2023
@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 1, 2023
@ti-chi-bot ti-chi-bot bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 5, 2023
@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 5, 2023

/rebuild

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 15, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-06-15 03:53:13.481581834 +0000 UTC m=+230589.896185913: ☑️ agreed by JaySon-Huang.
  • 2023-06-15 04:53:06.759672915 +0000 UTC m=+234183.174277005: ☑️ agreed by flowbehappy.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jun 15, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: flowbehappy, JaySon-Huang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [JaySon-Huang,flowbehappy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Lloyd-Pottiger
Lloyd-Pottiger reviewed Jun 15, 2023
View reviewed changes
hongyunyan and others added 5 commits June 15, 2023 14:19
JaySon-Huang
JaySon-Huang reviewed Jun 15, 2023
View reviewed changes
hongyunyan and others added 3 commits June 15, 2023 15:19
@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 15, 2023

/merge

@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 15, 2023

/run-unit-test

@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 15, 2023

/merge

@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 15, 2023

/rebuild

@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 15, 2023

/run-integration-test

@hongyunyan
Copy link
Contributor Author

hongyunyan commented Jun 15, 2023

/run-unit-test

@ti-chi-bot ti-chi-bot bot merged commit 865ba65 into pingcap:master Jun 15, 2023
@JaySon-Huang JaySon-Huang mentioned this pull request Jul 17, 2023
Closed
@JaySon-Huang JaySon-Huang mentioned this pull request Dec 26, 2023
Closed
This was referenced Feb 23, 2024
Closed
Merged
Merged
@JaySon-Huang JaySon-Huang mentioned this pull request May 8, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

@Lloyd-Pottiger Lloyd-Pottiger Lloyd-Pottiger left review comments

@flowbehappy flowbehappy flowbehappy approved these changes

@lidezhu lidezhu Awaiting requested review from lidezhu

Assignees

No one assigned

Labels

approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

The threshold of schema apply diff/full is fixed and short

4 participants

@hongyunyan @flowbehappy @JaySon-Huang @Lloyd-Pottiger