Describe the bug, including details regarding any error messages, version, and platform.
Calling pyarrow.compute.struct_field crashes on some ChunkedArray.
versions:
Ubuntu 20.04, x86_64
Python: 3.8 and 3.11 (same behavior)
pyarrow: 10.0.1
Here is a reproducer:
Feather file triggering the crash:
crash.feather.gz
import pyarrow as pa
from pyarrow.compute import struct_field
import pyarrow.feather
path = "crash.feather"
table = pa.feather.read_table(path)
print(table)
# This crashes
struct_field(struct_field(struct_field(table[0], [3]), [2]), [1])
# And that crashes as well
struct_field(struct_field(struct_field(table[0], [3]), [3]), [0])
The backtrace in both cases is the same:
/arrow/cpp/src/arrow/array/array_nested.cc:686: Check failed: (data_->buffers[0]) == (nullptr)
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(+0xfc30c8)[0x7f852ed830c8]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(_ZN5arrow4util8ArrowLogD1Ev+0xed)[0x7f852ed8347d]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(_ZN5arrow16SparseUnionArray7SetDataESt10shared_ptrINS_9ArrayDataEE+0x154)[0x7f852efbb664]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(_ZN5arrow16SparseUnionArrayC1ESt10shared_ptrINS_9ArrayDataEE+0x53)[0x7f852efbb8c3]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(_ZN5arrow9MakeArrayERKSt10shared_ptrINS_9ArrayDataEE+0xe4f)[0x7f852eeb4d2f]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(_ZNK5arrow11StructArray17GetFlattenedFieldEiPNS_10MemoryPoolE+0x209)[0x7f852ef9ddc9]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(+0x974159)[0x7f852e734159]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(+0xef1f29)[0x7f852ecb1f29]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(+0xd9568e)[0x7f852eb5568e]
/lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/libarrow.so.1000(_ZNK5arrow7compute8Function7ExecuteERKSt6vectorINS_5DatumESaIS3_EEPKNS0_15FunctionOptionsEPNS0_11ExecContextE+0xf2)[0x7f852eb64612]
lisa/.lisa-venv-3.11/lib/python3.11/site-packages/pyarrow/_compute.cpython-311-x86_64-linux-gnu.so(+0x89d95)[0x7f8513983d95]
python[0x62cfe8]
python(PyObject_Vectorcall+0x35)[0x63a9d5]
python(_PyEval_EvalFrameDefault+0x76d)[0x58ab6d]
python[0x6dd49f]
python(PyEval_EvalCode+0x97)[0x6dd727]
python[0x6b7367]
python[0x6b73f0]
python[0x6b760b]
python(_PyRun_SimpleFileObject+0x194)[0x6b7bf4]
python(_PyRun_AnyFileObject+0x47)[0x6b7cb7]
python(Py_RunMain+0x2bc)[0x6af99c]
python(Py_BytesMain+0x2d)[0x6afb4d]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f8531ce2083]
python(_start+0x2e)[0x664e6e]
Component(s)
Python
Describe the bug, including details regarding any error messages, version, and platform.
Calling
pyarrow.compute.struct_fieldcrashes on someChunkedArray.versions:
Ubuntu 20.04, x86_64
Python: 3.8 and 3.11 (same behavior)
pyarrow: 10.0.1
Here is a reproducer:
Feather file triggering the crash:
crash.feather.gz
The backtrace in both cases is the same:
Component(s)
Python