-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
We have had a few issues / reports where the creation of DataFusionError during query planning has contributed significantly to the overall planning time. Specifically, #5309 and more recently #7522
At the core of the issue is that creating a DataFusionError is a fairly slow operation (as it requires allocating a new String to hold the error message, and sometimes, as in #5309, the construction of the message itself is significant.
Describe the solution you'd like
I would like to update the DataFusion codebase so it does not use Errors on the "happy path"
Tasks:
- Make a faster way to check column existence in optimizer (not
is_err()) #5309 - Optimize FilterExec::statistics / don't ignore errors #7553
Describe alternatives you've considered
We could potentially make it faster / cheaper to create DataFusion errors
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request