This is an umbrella issue for a variety of follow-up Jiras to refactor and improve the array kernels / function machinery to have less overhead and work more efficiently for parallel processing as well as small inputs (down to ~1000 elements per kernel invocation)

Reporter: Wes McKinney / @wesm

Related issues:

• [C++] Rewrite ExecuteScalarExpression to not use ScalarExecutor (is a parent of)
• [C++] Introduce initial ArraySpan, ExecSpan non-owning / shared_ptr-free data structures for kernel execution, refactor scalar kernels (is a parent of)
• [C++] Migrate VectorKernel implementations to use ExecSpan, ArraySpan (is a parent of)
• [C++] Migrate SCALAR_AGGREGATE, HASH_AGGREGATE functions to use ExecSpan (is a parent of)
• [C++] Remove ExecBatchIterator and usages thereof (is a parent of)
• [C++] Overhead of std::shared_ptr copies is causing thread contention (is related to)
• [C++] Avoid slicing array inputs in ExecBatchIterator that would result in one slice (is related to)
• [C++] Investigate performance regressions observed in Unique, VisitArraySpanInline (is related to)
• [C++] ArraySpan::IsNull/IsValid implementations are incorrect for union types (is related to)
• [C++] Determine desirable maximum length for ExecBatch in pipelined and parallel execution of kernels (is related to)
• [C++][Compute] Extract preallocation logic from KernelExecutor (is related to)
• [C++] (eventually) abandon scalar columns of an ExecBatch in favor of RLE encoded arrays (is related to)
• [C++] Refactor internal array sorting code to use ArraySpan (is related to)
• [C++][Compute] Improve memory efficiency in Grouper (is related to)
• [C++] Detect unauthorized memory allocations in function kernels (is related to)
• [C++] Remove "scalar" output modality from array kernels (is related to)
• [C++] arrow::compute::CallFunction needs a batch length for nullary functions (is related to)
• [C++] Improve kernel performance on small batches (is related to)

_{Note: This issue was originally created as ARROW-16755. Please see the migration documentation for further details.}