-
Notifications
You must be signed in to change notification settings - Fork 4k
Description
It would be good to formalize the design of kernels and the controlling query execution layer (e.g. volcano batch model?) to understand the following:
Contracts for kernels:
-
Thread safety of kernels?
-
When Kernels should allocate memory vs expect preallocated memory? How to communicate requirements for a kernels memory allocaiton?
-
How to communicate the whether a kernels execution is parallelizable across a ChunkedArray? How to determine if the order to execution across a ChunkedArray is important?
-
How to communicate when it is safe to re-use the same buffers and input and output to the same kernel?
What does the threading model look like for the higher level of control? Where should synchronization happen?
Reporter: Micah Kornfield / @emkornfield
Related issues:
Note: This issue was originally created as ARROW-4333. Please see the migration documentation for further details.