Commit 1943db3
committed
Read source arrays through a nanoarrow ArrowArrayView
Reading walked the ArrowArrays directly: hasNulls counted the bits of a
validity bitmap by hand where a producer had left null_count at -1, and
ListOffsets dispatched on the format string to read 32-bit ("+l") or
64-bit ("+L") offsets out of buffer 1, tracking the slot offset a sliced
array carries.
An ArrowArrayView does both. InputStream lays one out from the geometry
schema once and points it at each batch as it arrives, so the readers take
a view rather than a schema and array pair, hasNulls becomes a call to
ArrowArrayViewComputeNullCount, and the offsets come from
ArrowArrayViewListChildOffset, which picks its width from the view's
storage type. That function indexes the offsets buffer directly, so
listBegin still adds the view's own offset - the case that a new
test_sliced_arrays pins for both geometry types, and which fails without
it.
Pointing the view at a batch also checks the batch over, which is new: its
children and buffers must be the shape the schema promised and its offsets
must stay inside the arrays they index. A stream that declares linestrings
and then yields something else used to be read blindly through the
declared schema; it now raises, with the reason.
Verified bit-identical against the same golden arrays, with flat ru_maxrss
across the stream lifecycles and rejected sources.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqdWshmD4AUHhrqeMh86GS1 parent 6d438ea commit 1943db3
2 files changed
Lines changed: 155 additions & 158 deletions
0 commit comments