Commit c2ab64d
committed
Split geometries of any type through a new entry point
split_linestrings and split_polygons each take a column of one type and
give back GeoArrow of that type. A layer holding more than one has nowhere
to go: geopandas will not even write such a frame as geoarrow-encoded
("Geometry type combination is not supported"), so WKB is the only
encoding that can carry it - and until now WKB could only be read, not
written.
split_geometries takes each geometry on its own terms. A LineString or a
Polygon is split by the kernel for its type, a Point has nothing to split
and goes through unchanged, and a multi-part geometry or a
GeometryCollection is a container whose members are each handled the same
way, however deep they nest. Every piece is attributed to the row it came
out of, whatever it came out of. An empty geometry goes back as itself:
GEOMETRYCOLLECTION EMPTY stays that, rather than vanishing and taking its
row with it.
Almost none of that recursion is written here. geoarrow-c reports a
container's members as nested geom_starts carrying their own types, so
MixedSplitter only has to notice which level it is on and what the frame
it is closing turned out to hold - a frame that opened no children and
took no coordinates is an empty geometry, which is what tells
MULTILINESTRING EMPTY apart from a MULTILINESTRING with parts.
Pieces go out as geoarrow.wkb, since an Arrow stream has one schema for
every batch. The typed entry points keep their native zero-copy output
untouched, so only this path pays. The writer is driven through the same
visitor protocol the reader speaks, and a vector of Coord is already the
interleaved run a coords call describes, so nothing is copied writing
either - and the finished array is *moved* into the output struct rather
than rebuilt.
What that costs, on 20k linestrings splitting to 300k pieces, best of 9:
geoarrow in -> geoarrow out 19.4 ms 13.5 MB
wkb in -> geoarrow out 25.9 ms 13.5 MB
wkb in -> wkb out 24.1 ms 16.2 MB
So most of the difference is decoding WKB on the way in, not writing it on
the way out, and the blobs are about 20% larger than the buffers. A column
that really does hold one type should still use the typed functions, and
the docstring says so.
A mixed split of a single-type column gives exactly the pieces the typed
split gives, for both geometry types and from either encoding - the tests
assert that pairing, since the typed path is meant to be an optimisation
rather than a variant. Native path bit-identical, ru_maxrss flat across
fourteen stream lifecycles.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqdWshmD4AUHhrqeMh86GS1 parent 11767df commit c2ab64d
3 files changed
Lines changed: 662 additions & 48 deletions
0 commit comments