Enhancement

1. Since there is no unit test for the Interpreter module, Interpreter test framework aims to supplement the content of the unit test, improve the stability of the module, and find more hidden bugs at home.
2. TiFlash planner needs to rely on unit tests to ensure the quality and progress of development during the refactoring process.
3. User can mock input columns and feed them into the "Execution streams(BlockInputStreams in TiFlash)", then they can test executors(except MPP related logic) of TiFlash locally.
4. Further more, MPP related tests will be supported.
5. When developing more features in the future, good unit testing can be used as the basis for development to further improve the efficiency and quality of code development.

Non MPP tests development progress:

• Split AstToExecutor from dbgFuncCoprocessor. #4610
• Interpreter: Hand write tipb::Executor. #4632
• Interpreter: Mock exchange executors. #4706
• Interpreter: Serialize executors #4742
• Interpreter: Run interpreter test without exchange executors #4788
• Interpreter: Print BlockInputStream #4911
• Interpreter: Run interpreter test with exchange executors #4858
• Test: Mock Input columns for operator tests #5041
• Test: Mock window function, refactor window function tests #5021
• Test framework improvement #5262
• Replace initializer_list with vector for planner test framework #5295
• Print content of columns for gtest #5243
• Test Aggregation functions #5347
• Support more Aggregation functions to test #5345
• Support collation for planner refactory test framwork #5432
• Support alias for planner refactory test framework #5469
• Failed to use qualified format to reference table column in executor test framework #5510
• Add UT for list struct based executors #5534
• Fix test framework's groupby expression support #5419
• Enhance join test framework with conditional expressions #5543
• Enhance executor join tests #5305
• Test: refine Executor Unit Test #5889 add test log when failed.
• Test: Support coprocessor test #5957
• Test: wrap some API for aggregation function tests #5787
• Test: Refine astToExecutor #5895
• Test: refine astToExecutor to ExecutorBinders #5915
• Test more types of executors[WIP].
• Support more expressions to test.
• More user friendly exception throw.
• Check the legality and correctness of user input.
• Add some easy to use template to write tests.
• When result is wrong, print more user friendly message to help debug.
• Add some built-in input columns for user to generate a bunch of tests.
• Generate columns with different types and size. Test: RandomGenColumns #5743
• Generate columns following different data distribution.
• Support regression test like this https://github.com/risinglightdb/sqlplannertest-rs. Here is the detail A more friendly way to update the unit test cases in gtest_interpreter.cpp #6676
• A document to demonstrate how to use the test framework.
• Bind test framework with google bench.
• New executor should implement Serializer. New BlockInputStream should implement appendInfo().
• Improve the speed of tests.
• A test generator like sqlsmith.(Just for fun)
• Test: port some of delta-merge test into ut, part 1 #6160
• Test: Bind delta merge storage with executor ut #6561
• Test: support mock not null column #7059

MPP tests development progress:

• *: Test DagRequest to MPPTasks #5369
• Test: Mock TiFlash compute service and dispatch MPPTask to single service. #5450
• Test: Mock multiple compute service to run MPPTasks. #5573
• Orchestrate MPP Tasks.
• Get task status of each TiFlash service. Test: Test CancelMPPTask in UT #5732
• Mock Storage. Test: Mock multiple compute service to run MPPTasks. #5573
• Test: Test CancelMPPTask in UT #5732
• Write TPCH queries in the framework!

Run with storage layer

• MockStorage with StorageDeltaMerge. Test: Bind delta merge storage with executor ut #6561
• Support push down filter test Test: support push down filter in ut #6573

Feel free to ask me anything about the Test Framework.