-
Notifications
You must be signed in to change notification settings - Fork 4k
Open
Labels
Component: C++Status: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwiseIssues and PRs flagged as stale which are due to be closed if no indication otherwiseType: enhancement
Description
ARROW-11591 adds Grouper for identifying groups based on multiple key columns.
For a large number of groups, it is beneficial to do a first pass partitioning on the key columns so that each worker thread only handles a subset of the query's groups. This is usually accomplished by computing only the hashes of the keys (not full group identity) and pushing slices of the input batches to workers based on those.
This would probably make sense as a member function of Grouper, maybe Grouper::ConsumeImperfect
Reporter: Ben Kietzman / @bkietz
Related issues:
- [C++] Query engine umbrella issue (is a child of)
Note: This issue was originally created as ARROW-12044. Please see the migration documentation for further details.
Metadata
Metadata
Assignees
Labels
Component: C++Status: stale-warningIssues and PRs flagged as stale which are due to be closed if no indication otherwiseIssues and PRs flagged as stale which are due to be closed if no indication otherwiseType: enhancement