Reduce redundant pack reads during building bitmap filter for delta merge case#9876
Conversation
Signed-off-by: gengliqi <gengliqiii@gmail.com>
Signed-off-by: gengliqi <gengliqiii@gmail.com>
Signed-off-by: gengliqi <gengliqiii@gmail.com>
5cefac1 to
3658763
Compare
Signed-off-by: gengliqi <gengliqiii@gmail.com>
Co-authored-by: Lloyd-Pottiger <60744015+Lloyd-Pottiger@users.noreply.github.com>
112e089 to
e36245d
Compare
e36245d to
b00df5f
Compare
|
If there is a delete mark in the column file, it seems this situation cannot be directly skipped. |
tiflash/dbms/src/Storages/DeltaMerge/File/DMFilePackFilter.cpp Lines 342 to 349 in 82932c3 If there is any delete mark in the dmfile, then |
[LGTM Timeline notifier]Timeline:
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JaySon-Huang, Lloyd-Pottiger The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
|
In response to a cherrypick label: new pull request created to branch |
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
What problem does this PR solve?
Issue Number: close #9875
Problem Summary:
See #9875
This optimization can provide a significant performance boost when the data being read is not updated frequently.
What is changed and how it works?
Run CHBenchmark 1500: Overall 26.5% Improvement
The above table shows the delta rows for each table in
chbench_deltadatabase.All Stable Performance Comparison
The following table presents the performance improvements for All Stable compared to both the original version (with Delta) and the optimized version (with Delta):
The chbenchmark data was inserted into another database. Then the compact table command is executed. As a result, the data might differ. So there are some differences in the query plans. For example, in Q20, the query plan uses hash partition join and includes late materialization for order_line table in the All Stable version, while original version uses broadcast join and no late materialization for order_line table. This causes Q20 to run slower in All Stable compared to the original version due to exchange overhead and low selectivity.
Check List
Tests
Side effects
Documentation
Release note