Skip to content

[VL] pick split with most data prefetched #11821

@FelixYBW

Description

@FelixYBW

Description

Currently in Velox, we pick the first split where its stete is KPrepared, not the one with most data fetched. Once it's picked, we will need to wait until its data return later, even the other splits already have ready data.

Ideally we should wait there until the first split has its first rowgroup data returned, but the logic is too complex. Instead we can pick the split which has most data returned in first rowgroup.

Gluten version

None

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions