Skip to content

[C++] Sketch out design for kernels and "query" execution in compute layer #20902

@asfimport

Description

@asfimport

It would be good to formalize the design of kernels and the controlling query execution layer (e.g. volcano batch model?) to understand the following:

Contracts for kernels:

  • Thread safety of kernels?

  • When Kernels should allocate memory vs expect preallocated memory?  How to communicate requirements for a kernels memory allocaiton?

  • How to communicate the whether a kernels execution is parallelizable across a ChunkedArray?  How to determine if the order to execution across a ChunkedArray is important?

  • How to communicate when it is safe to re-use the same buffers and input and output to the same kernel?

    What does the threading model look like for the higher level of control?  Where should synchronization happen?

Reporter: Micah Kornfield / @emkornfield

Related issues:

Note: This issue was originally created as ARROW-4333. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions