safe writes for array

[binary-signal]

Purpose

Linked issue: close #2164

Fix IndexOutOfBoundsException when writing rows with array columns where the total number of array elements exceeds INITIAL_CAPACITY (1024) while the row count stays below it.

Brief change log

In ArrowWriter.writeRow(), the handleSafe flag is determined by comparing row count against INITIAL_CAPACITY:

boolean handleSafe = recordsCount >= INITIAL_CAPACITY;

When handleSafe = false, Arrow writers use vector.set() which doesn't auto-grow the buffer. The bug is in ArrowArrayWriter.doWrite() which passes the parent's handleSafe flag to the element writer. However, array element indices grow based on cumulative element count, not row count.

Example: 250 rows with 10-element arrays → row count (250) < 1024 so handleSafe = false, but total elements (2500) exceeds the vector's initial capacity, causing IndexOutOfBoundsException.

Fix:
Always use safe writes (handleSafe = true) for array element writers in ArrowArrayWriter.doWrite(), since element indices can exceed INITIAL_CAPACITY independently of row count.

// Before
elementWriter.write(fieldIndex, array, arrIndex, handleSafe);

// After
elementWriter.write(fieldIndex, array, arrIndex, true);

Tests

• Added ArrowReaderWriterTest#testArrayWriterWithManyElements: writes 200 rows with 10-element arrays (2000 total elements), verifying serialization succeeds and data can be read back correctly.

API and Format

No API or storage format changes.

Documentation

No documentation changes needed. This is a bug fix.

[rionmonster]

LGTM — looks pretty straightforward. Approved! 👍

[XuQianJin-Stars]

+1

[vamossagar12]

LGTM

[vamossagar12]

We should mention in the comments on the class that the handleSafe field is ignored when writing.