[C++] Take on string chunked arrays slow and fails #26738
Description
import pyarrow as pa
a = pa.array(['a'] * 2**26)
c = pa.chunked_array([a] * 2*18)
c.take([0, 1])Gives
----------------------------------------
ArrowInvalidTraceback (most recent call last)
<ipython-input-4-57099ee02815> in <module>
----> 1 c.take([0, 1])
~/github/apache/arrow/python/pyarrow/table.pxi in pyarrow.lib.ChunkedArray.take()
~/github/apache/arrow/python/pyarrow/compute.py in take(data, indices, boundscheck, memory_pool)
421 """
422 options = TakeOptions(boundscheck=boundscheck)
--> 423 return call_function('take', [data, indices], options, memory_pool)
424
425
~/github/apache/arrow/python/pyarrow/_compute.pyx in pyarrow._compute.call_function()
~/github/apache/arrow/python/pyarrow/_compute.pyx in pyarrow._compute.Function.call()
~/github/apache/arrow/python/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()
~/github/apache/arrow/python/pyarrow/error.pxi in pyarrow.lib.check_status()
ArrowInvalid: offset overflow while concatenating arrays
PS: did not check master but 3.0.0.dev238+gb0bc9f8d
Reporter: Maarten Breddels / @maartenbreddels
Related issues:
- [C++] Take kernel can't handle ChunkedArrays that don't fit in an Array #25822 (duplicates)
- [Python] pyarrow.concat_arrays segfaults if a resulting StringArray's capacity overflows #26180 (relates to)
Note: This issue was originally created as ARROW-10799. Please see the migration documentation for further details.