Skip to content

chore: Revise batch pull approach to more follow C Data interface semantics#893

Merged
viirya merged 3 commits intoapache:mainfrom
viirya:revise
Sep 3, 2024
Merged

chore: Revise batch pull approach to more follow C Data interface semantics#893
viirya merged 3 commits intoapache:mainfrom
viirya:revise

Conversation

@viirya
Copy link
Copy Markdown
Member

@viirya viirya commented Aug 30, 2024

Which issue does this PR close?

Part of #885.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

} else {
val localTopK = if (orderingSatisfies) {
CometExecUtils.getNativeLimitRDD(childRDD, output, limit)
CometExecUtils.getNativeLimitRDD(childRDD, child.output, limit)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix in CometTakeOrderedAndProjectExec is at #896. I included it to verify it can make this PR passing CI.

@viirya
Copy link
Copy Markdown
Member Author

viirya commented Sep 2, 2024

The last test failure is fixed by #902.

@viirya viirya merged commit 046a62c into apache:main Sep 3, 2024
@viirya
Copy link
Copy Markdown
Member Author

viirya commented Sep 3, 2024

Thanks @andygrove @huaxingao

@viirya viirya deleted the revise branch September 3, 2024 15:17
coderfender pushed a commit to coderfender/datafusion-comet that referenced this pull request Dec 13, 2025
…antics (apache#893)

* chore: Revise batch pull approach to more follow C Data interface semantics

* fix clippy

* Remove ExportedBatch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants