Enhancement

• Currently, there is no interface to get the memory usage of plan cache. We can only estimate the memory each plan used and set a limit number to control the memory.It is imprecise and may cause OOM of TiDB.So we would like to add memory tracker for plan cache to count and manage the memory usage.

Items

PhysicalPlan memory trace

• Expression(Column/Constant/CorrelatedColumn/ScalarFunction) memory trace planner: Expression memory trace #37624
• PhysicalPlan(sort operator) memory trace planner: PhysicalPlan memory trace 1 #37655
• PhysicalPlan(scan operator) memory trace planner: PhysicalPlan memory trace 2 #37668
• PhysicalPlan(agg/limit/select/projection operator) memory trace planner: PhysicalPlan memory trace 3 #37731
• PhysicalPlan(reader operator) memory trace planner: PhysicalPlan memory trace 4 #37736
• PhysicalPlan(join operator) memory trace planner: PhysicalPlan memory trace 5 #37860
• PhysicalPlan(pointGet/apply/topN operator) memory trace planner: PhysicalPlan memory trace 6 #37841
• PhysicalPlan(mock/CTE ) memory trace planner: PhysicalPlan memory trace 7 #38030
• PhysicalPlan(SimpleWrapper/MemTable/ExchangeReceiver/ExchangeSender/Lock ) memory trace planner: PhysicalPlan memory trace 8 #38162
• PhysicalPlan (PhysicalMaxOneRow/TableDual/Window/Shuffle/ShuffleReceiverStub/Show/ShowDDLJobs) memory trace planner: PhysicalPlan memory trace 9 #38220

Count other memory usage in plan cache

• LRUPlanCache、planCacheKey、PlanCacheValue memory trace planner: LRUPlanCache memory trace #38069 1
• Plan(Insert/Update/Delete) memory trace planner: Plan in PlanCacheValue memory trace #38357
• PlanCacheStmt、ParamMarker memory trace
• LRU according to memory usage 3
• task(copTask/mppTask/rootTask) memory trace planner: supply Task memory usage and correct some accumulation #38638

Test and attach

• Test accuracy of plan cache memory trace and correct
• Access memTracker (daily-run test)
• Conform the security of plan cache memory trace(No panic)
  • nil pointer planner: judge nil pointer before call MemoryUsage #38764
  • add switch control the memory monitor planner: add switch for plan cache memory trace #38824

Others

• Add monitor (mem usage) 2 planner: add monitor for prepared plan cache memory usage  #38507
• Add monitor (plan number and others...) 4 planner: add monitor for prepared plan cache memory usage  #38507
• Supply document
  • Add descriptions of new panels and update the plan cache doc docs-cn#11799
  • planner: add description for plan cache variable docs-cn#11861