Skip to content

[Proposal] Remove idle conjuncts in olap scan node which had alreadly push to Storage Engine #4986

@HappenLee

Description

@HappenLee

Motivation

Now, Doris's OLAP_SCAN_NODE can push the conjuncts condition to the storage engine.
But after push it down to storage engine, the conditions still need to be filtered in the OLAP_SCANNER.

It's is a very unreasonable way. Therefore, we need to figure out the filtering conditions that have been pushed down to the storage engine. Delete these pushed down conditions on the OLAP SCANNER. We should not waste CPU resources on the conditions that have been filtered.

It can bring us the following benefits:

I Had Do POC in the data of SSB:

select * from customer where C_NATION = "CHINA" order by C_CUSTKEY limit 10, 15; 

There are 3000W rows of data

Cost Time Origin Doris Delete Push Down Conjuncts
5.2s 4.3s

It's about 20% performance improvement

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions