-
Notifications
You must be signed in to change notification settings - Fork 270
Closed
apache/datafusion
#18875Labels
Description
Describe the bug
When running TPC-DS benchmarks against 100 GB data set I see a large regression in performance. For example, here are the timings for q72 before and after adding support for SMJ with join condition.
Adding support for SMJ with join condition means that more of the plan is likely running natively and the performance issue isn't necessarily directly related to SMJ.
before
"72": [
22.729433059692383,
18.11495876312256,
17.545786142349243
]
after
"72": [
38.576566219329834,
35.433213233947754,
35.262585401535034
]
A secondary issue is that I do not see metrics for CometSort / CometSortMergeJoin.
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response
