Skip to content

*: move config file option mem-quota-query to sysvar#34141

Merged
ti-chi-bot merged 27 commits intopingcap:masterfrom
espresso98:tidb_mem_quota_query
Apr 25, 2022
Merged

*: move config file option mem-quota-query to sysvar#34141
ti-chi-bot merged 27 commits intopingcap:masterfrom
espresso98:tidb_mem_quota_query

Conversation

@espresso98
Copy link
Collaborator

@espresso98 espresso98 commented Apr 20, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #33769

Problem Summary:

The option mem-quota-query has historically been a config option and tidb-mem-quota-query already exists as the only session variable. Based on requirements from cloud & PM it should instead be a GLOBAL sysvar.

What is changed and how it works?

This does the minimum work to convert mem-quota-query to a global (cluster-wide) sysvar, and removes the config option (placing it in the previously deprecated list).

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

The previous default (TiDB 2.1, 3.1) for memory quota was 32G. When upgrading from TiDB 2.1/3.0 to 4.0 the 32G limit is preserved in upgrade, so I was concerned that this might create an upgrade problem for some users.

However, if you manually try an upgrade from v2.1/3.0 -> 4.0 -> 5.0, the 32G default is lost and 1G is used. So the intended compatibility feature is not working correctly. This is why the test TestIssue20900_1 was removed -- it does not work past version 4.0 any more. Because the default_memory_quota_query is deleted from mysql.tidb it is also not possible to fix this in a new bootstrap task. The new default is always picked up.

  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

The TiDB configuration file option `mem-quota-query` has been replaced by the global system variable `tidb-mem-quota-query`. This change makes it easier to modify this setting on a cluster wide basis.

@espresso98 espresso98 requested a review from a team as a code owner April 20, 2022 17:24
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Apr 20, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • morgo
  • ngaut

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 20, 2022
@espresso98
Copy link
Collaborator Author

espresso98 commented Apr 20, 2022

Please review @morgo

@sre-bot
Copy link
Contributor

sre-bot commented Apr 20, 2022 •
edited
Loading

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/f1fdc4775c5ec02828fefe7ba17c42184a0a66dc

@morgo morgo self-requested a review April 20, 2022 19:18
@morgo morgo added the compatibility-breaker Violation of forwards/backwards compatibility in a design-time piece. label Apr 20, 2022
morgo
morgo reviewed Apr 20, 2022
View reviewed changes
@morgo morgo self-requested a review April 21, 2022 00:35
@ti-chi-bot ti-chi-bot added status/LGT1 Indicates that a PR has LGTM 1. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 21, 2022
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 23, 2022
@hawkingrei
Copy link
Member

hawkingrei commented Apr 24, 2022

@espresso98 Please fix the conflict.

AilinKid and others added 13 commits April 25, 2022 11:41
@AilinKid @espresso98
ddl: fix the partial-count collection for ddl reorg encountering erro…
36a5b09 
…rs halfway (pingcap#34165)

ref pingcap#32306, close pingcap#34164
@AndreMouche @espresso98
executor/infoschema_reader.go: skip the unidentifiable table-attribut…
db12314 
…es (pingcap#34129)

close pingcap#33665
@hawkingrei @espresso98
*: remove useless code (pingcap#34159)
843ba30 
ref pingcap#31716
@hawkingrei @espresso98
ddl: add ddlCtx for ddl interface (pingcap#34153)
9c5cae0 
ref pingcap#33635
@hawkingrei @espresso98
parser: turn some functions into member functions of model.Job (pingc…
cda056c 
…ap#34175)

ref pingcap#31716
@TonsnakeLin @espresso98
Doc: Add a design doc for optimizing gc safepoint advance (pingcap#32934
4c2f468 
)

close pingcap#32725
@tiancaiamao @espresso98
executor: flush statement should trigger implicit commit (pingcap#34134)
dc48849 
close pingcap#34180
@3pointer @espresso98
br_gcs: fix tikv integration test (pingcap#34191)
7dc7ef8 
close pingcap#33811
@fanrenhoo @espresso98
variable: make lc_messages read only (pingcap#33708)
eb556e3 
close pingcap#33707
@ekexium @espresso98
session: add some compatibility checks for non-transactional DML (pin…
bbceabf 
…gcap#34127)

ref pingcap#33485
@wjhuang2016 @espresso98
ddl: rewrite some cancel tests to an unified test (pingcap#34157)
2ae91aa 
close pingcap#32939
@lcwangchao @espresso98
*: Move stale read context to TxnManager (pingcap#33812)
e4fe87c 
close pingcap#31954, close pingcap#33014, close pingcap#33832
@wjhuang2016 @espresso98
*: refine some tests for concurrent DDL (pingcap#34215)
37dcc42 
ref pingcap#33008
@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 25, 2022
@espresso98 espresso98 force-pushed the tidb_mem_quota_query branch from c257548 to 37dcc42 Compare April 25, 2022 19:15
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 25, 2022
@morgo
Copy link
Contributor

morgo commented Apr 25, 2022

/merge

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Apr 25, 2022

This pull request has been accepted and is ready to merge.

DetailsCommit hash: f1fdc47

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 25, 2022
@ti-chi-bot ti-chi-bot merged commit 20a5819 into pingcap:master Apr 25, 2022
@sre-bot
Copy link
Contributor

sre-bot commented Apr 25, 2022

TiDB MergeCI notify

✅ Well Done! New fixed [1] after this pr merged.

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-common-test ✅ all 11 tests passed 9 min 52 sec Fixed
idc-jenkins-ci-tidb/common-test 🟢 all 12 tests passed 5 min 54 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1 🟢 all 26 tests passed 5 min 31 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 5 min 25 sec Existing passed
idc-jenkins-ci-tidb/integration-ddl-test 🟢 all 6 tests passed 5 min 14 sec Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 5 min 3 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 37 sec Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test 🟢 all 1 tests passed 2 min 54 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed

This was referenced May 2, 2022
Open
Closed
@CbcWestwolf CbcWestwolf mentioned this pull request May 9, 2022
Closed
14 tasks
@espresso98 espresso98 mentioned this pull request May 11, 2022
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morgo morgo morgo approved these changes

@ngaut ngaut ngaut approved these changes

Assignees

No one assigned

Labels

compatibility-breaker Violation of forwards/backwards compatibility in a design-time piece. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.