-
Notifications
You must be signed in to change notification settings - Fork 270
Closed
Labels
Description
What is the problem the feature request solves?
When native shuffle has a single output partition, we append the rows from the input batch into array builders and then create a new batch, which is identical to the input batch. We could skip all of this processing and just pass the batch though unmodified.
Partitioning::UnknownPartitioning(n) if *n == 1 => {
let buffered_partitions = &mut self.buffered_partitions;
assert!(
buffered_partitions.len() == 1,
"Expected 1 partition but got {}",
buffered_partitions.len()
);
let indices = (0..input.num_rows()).collect::<Vec<usize>>();
self.append_rows_to_partition(input.columns(), &indices, 0)
.await?;Describe the potential solution
No response
Additional context
No response