Enhancement

background

Currently tiflash's interpreter is designed based on DAGQueryBlock.
DAGQueryBlock is designed to coprocess read, and it can handle simple queries well.
However, there is no way to handle the complex query of MPP, so the DAGQueryBlock tree is introduced, which is a compromise that makes the code more complicated and affects the subsequent iterative development.
We need to refactor the interpreter code in a plan visit way.

design

Therefore we introduce a new abstraction layer called PhysicalPlan to replace DAGQueryBlock.
It will be responsible for the processing of ExpressionAction and the generation of BlockInputStream.
We call this organization of code the planner.

DAGRequest (executor tree or executor array).
     |
     |
     v
physical plan <---+
     |            |
     |            | optimize
     |            |
     +------------+   
     |
     |
     v
BlockInputStream

why planner

Most databases are designed based on planner.

• Friendly for database developers to understand code.
• We can improve tiflash with a general approach from the DB area, such as pipeline mode.

todo list

• public desgin doc
  • doc: planner refactor #4743
• Small refactoring of Interpreter before planner refactoring
  • Improve the code of Interpreter #4118
• interpreter test framework
  • Interpreter & Executor & MPP unit test framework #4609
• planner refactory
  • poc [DNM] Physical plan #4601
  • Introduce PhysicalPlan/planner, add switch and set it to planner by default
    • Introduce physical plan and add switch #4820
  • Replace all operators with PhysicalPlan
    • common plans: projection/agg/filter/exchange_(sender/receiver)/limit/topn
      • Planner: Support common physical plans #5143
    • window function
      • Planner: support window #5363
    • join
      • Planner: support Join #5320
    • table scan
      • Planner: support table scan #5335
  • remove DAGQueryBlock
    - Planner: planner without DAGQueryBlock #5381
• merge to master
  • feature branch: https://github.com/pingcap/tiflash/tree/planner_refactory
  • Feature Branch: merge master to planner_refactory #5493
  • Planner: Support Planner Interpreter #5321