Documentation update for geoarrow-array crate (#1027)

- Write crate-level documentation for `geoarrow-array`.
- At least have a minimal docstring for every module
- Add `from_geoarrow_array` top-level function to convert from a `&dyn
Array`, `&Field` pair into an `Arc<dyn GeoArrowArray>`
- Warn (which will error on CI) on missing docs in the geoarrow-array
crate.

Author: kylebarron

Cargo.lock

@@ -19,6 +19,7 @@ arrow-buffer = "54.3.1"
 arrow-schema = "54.3.1"
 geo = "0.30.0"
 geo-traits = "0.2.0"
+geo-types = "0.7.16"
 num-traits = "0.2.19"
 rstar = "0.12.2"
 serde = "1"

Cargo.toml

@@ -25,4 +25,5 @@ wkb = { workspace = true }
 wkt = { workspace = true }
 
 [dev-dependencies]
+geo-types = { workspace = true }
 geo = { workspace = true }

rust/geoarrow-array/Cargo.toml

@@ -1 +1,110 @@
 # geoarrow-array
+
+The central type in Apache Arrow are arrays, which are a known-length sequence of values all having the same type. This crate provides concrete implementations of each type defined in the [GeoArrow specification], as well as a [GeoArrowArray] trait that can be used for type-erasure.
+
+[GeoArrow specification]: https://github.com/geoarrow/geoarrow
+
+In order to minimize overhead of dynamic downcasting, the array types in this crate are defined "natively" and there's a `O(1)` conversion process that needs to happen to convert between a GeoArrow array type and an [`arrow`][arrow_array] array type.
+
+## Building a GeoArrow Array
+
+Use [builders][builder] to construct GeoArrow arrays. These builders offer a push-based interface to construct arrays from a series of objects that implement [`geo-traits`][geo_traits].
+
+```rust
+# use geo_traits::{CoordTrait, PointTrait};
+# use geoarrow_array::array::PointArray;
+# use geoarrow_array::builder::PointBuilder;
+# use geoarrow_array::scalar::Point;
+# use geoarrow_array::ArrayAccessor;
+# use geoarrow_schema::{CoordType, Dimension, PointType};
+#
+let point_type = PointType::new(CoordType::Separated, Dimension::XY, Default::default());
+let mut builder = PointBuilder::new(point_type);
+
+builder.push_point(Some(&geo_types::point!(x: 0., y: 1.)));
+builder.push_point(Some(&geo_types::point!(x: 2., y: 3.)));
+builder.push_point(Some(&geo_types::point!(x: 4., y: 5.)));
+
+let array: PointArray = builder.finish();
+
+let point_0: Point<'_> = array.get(0).unwrap().unwrap();
+assert_eq!(point_0.coord().unwrap().x_y(), (0., 1.));
+```
+
+Converting a builder to an array via `finish()` is always `O(1)`.
+
+## Converting to and from [`arrow`][arrow_array] Arrays
+
+The `geoarrow` crates depend on and are designed to be used in combination with the upstream [Arrow][arrow_array] crates. As such, we have easy integration to convert between representations of each crate.
+
+Note that an [`Array`] or [`ArrayRef`] only maintains information about the physical [`DataType`] and will lose any extension type information. Because of this, it's **imperative to store an [`Array`] and [`Field`] together** since the [`Field`] persists the Arrow [extension metadata]. A [`RecordBatch`] holds an [`Array`] and [`Field`] together for each column, so a [`RecordBatch`] will persist extension metadata.
+
+### Converting to GeoArrow Arrays
+
+If you have an [`Array`] and [`Field`] but don't know the geometry type of the array, you can use [`from_arrow_array`][array::from_arrow_array]:
+
+```rust
+# use std::sync::Arc;
+#
+# use arrow_array::Array;
+# use arrow_schema::Field;
+# use geoarrow_array::array::{from_arrow_array, PointArray};
+# use geoarrow_array::cast::AsGeoArrowArray;
+# use geoarrow_array::{GeoArrowArray, GeoArrowType};
+#
+fn use_from_arrow_array(array: &dyn Array, field: &Field) {
+    let geoarrow_array: Arc<dyn GeoArrowArray> = from_arrow_array(array, field).unwrap();
+    match geoarrow_array.data_type() {
+        GeoArrowType::Point(_) => {
+            let array: &PointArray = geoarrow_array.as_point();
+        }
+        _ => todo!("handle other geometry types"),
+    }
+}
+```
+
+If you know the geometry type of your array, you can use one of its `TryFrom` implementations to convert directly to that type. This means you don't have to downcast on the GeoArrow side from an `Arc<dyn GeoArrowArray>`.
+
+```rust
+# use arrow_array::Array;
+# use arrow_schema::Field;
+# use geoarrow_array::array::PointArray;
+#
+fn convert_to_point_array(array: &dyn Array, field: &Field) {
+    let point_array = PointArray::try_from((array, field)).unwrap();
+}
+```
+
+### Converting to [arrow][arrow_array] Arrays
+
+You can use the [`to_array_ref`][GeoArrowArray::to_array_ref] or [`into_array_ref`][GeoArrowArray::into_array_ref] methods on [`GeoArrowArray`] to convert to an [`ArrayRef`].
+
+Alternatively, if you have a concrete GeoArrow array type, you can use [`IntoArray`] to convert to a concrete arrow array type.
+
+The easiest way today to access an arrow [`Field`] is to use [`IntoArray::ext_type`] and then call `to_field` on the result. We like to make this process simpler in the future.
+
+## Downcasting a GeoArrow array
+
+Arrays are often passed around as a dynamically typed `&dyn GeoArrowArray` or [`Arc<dyn GeoArrowArray>`][GeoArrowArray].
+
+While these arrays can be passed directly to compute functions, it is often the case that you wish to interact with the concrete arrays directly.
+
+This requires downcasting to the concrete type of the array. Use the [`cast::AsGeoArrowArray`] extension trait to do this ergonomically.
+
+```rust
+use geoarrow_array::cast::AsGeoArrowArray;
+use geoarrow_array::{ArrayAccessor, GeoArrowArray};
+
+fn iter_line_string_array(array: &dyn GeoArrowArray) {
+    for row in array.as_line_string().iter() {
+        // do something with each row
+    }
+}
+```
+
+[`Array`]: arrow_array::Array
+[`ArrayRef`]: arrow_array::ArrayRef
+[`DataType`]: arrow_schema::DataType
+[`Field`]: arrow_schema::Field
+[`RecordBatch`]: arrow_array::RecordBatch
+[extension metadata]: https://arrow.apache.org/docs/format/Columnar.html#format-metadata-extension-types

rust/geoarrow-array/README.md

@@ -10,7 +10,10 @@ use crate::builder::InterleavedCoordBufferBuilder;
 use crate::error::{GeoArrowError, Result};
 use crate::scalar::InterleavedCoord;
 
-/// A an array of coordinates stored interleaved in a single buffer.
+/// An array of coordinates stored interleaved in a single buffer.
+///
+/// This stores all coordinates in interleaved fashion in a single underlying buffer: e.g. `xyxyxy`
+/// for 2D coordinates.
 #[derive(Debug, Clone, PartialEq)]
 pub struct InterleavedCoordBuffer {
     pub(crate) coords: ScalarBuffer<f64>,

rust/geoarrow-array/src/array/coord/interleaved.rs

@@ -12,10 +12,10 @@ use crate::error::{GeoArrowError, Result};
 use crate::scalar::SeparatedCoord;
 use geo_traits::CoordTrait;
 
-/// The GeoArrow equivalent to `Vec<Option<Coord>>`: an immutable collection of coordinates.
+/// An array of coordinates stored in separate buffers of the same length.
 ///
-/// This stores all coordinates in separated fashion as multiple underlying buffers: `xxx` and
-/// `yyy`.
+/// This stores all coordinates in separated fashion as multiple underlying buffers: e.g. `xxx` and
+/// `yyy` for 2D coordinates.
 #[derive(Debug, Clone, PartialEq)]
 pub struct SeparatedCoordBuffer {
     /// We always store a buffer for all 4 dimensions. The buffers for dimension 3 and 4 may be

rust/geoarrow-array/src/array/coord/separated.rs

@@ -18,6 +18,8 @@ use crate::error::{GeoArrowError, Result};
 use crate::scalar::Geometry;
 use crate::trait_::{ArrayAccessor, GeoArrowArray, IntoArrow};
 
+/// An immutable array of geometries of unknown geometry type and dimension.
+///
 /// # Invariants
 ///
 /// - All arrays must have the same dimension

rust/geoarrow-array/src/array/geometry.rs

@@ -16,7 +16,7 @@ use crate::scalar::GeometryCollection;
 use crate::trait_::{ArrayAccessor, GeoArrowArray, IntoArrow};
 use crate::util::{offsets_buffer_i64_to_i32, OffsetBufferUtils};
 
-/// An immutable array of GeometryCollection geometries using GeoArrow's in-memory representation.
+/// An immutable array of GeometryCollection geometries.
 ///
 /// This is semantically equivalent to `Vec<Option<GeometryCollection>>` due to the internal
 /// validity bitmap.

rust/geoarrow-array/src/array/geometrycollection.rs

@@ -16,7 +16,7 @@ use arrow_buffer::{NullBuffer, OffsetBuffer};
 use arrow_schema::{DataType, Field};
 use geoarrow_schema::{LineStringType, Metadata};
 
-/// An immutable array of LineString geometries using GeoArrow's in-memory representation.
+/// An immutable array of LineString geometries.
 ///
 /// This is semantically equivalent to `Vec<Option<LineString>>` due to the internal validity
 /// bitmap.

rust/geoarrow-array/src/array/linestring.rs

@@ -1,57 +1,6 @@
-//! Implementations of immutable GeoArrow arrays plus builders to more easily create arrays.
-//!
-//! There are three primary types of structs in this module: arrays, builders, and capacity
-//! counters.
-//!
-//! ## Arrays
-//!
-//! Arrays
-//!
-//! These arrays implement the binary layout defined in the [GeoArrow specification](https://github.com/geoarrow/geoarrow).
-//!
-//!
-//!
-//! These include:
-//!
-//! - [`PointArray`]
-//! - [`LineStringArray`]
-//! - [`PolygonArray`]
-//! - [`MultiPointArray`]
-//! - [`MultiLineStringArray`]
-//! - [`MultiPolygonArray`]
-//! - [`GeometryArray`]
-//! - [`GeometryCollectionArray`]
-//! - [`RectArray`]
-//!
-//! ## Builders
-//!
-//! Builders are designed to make it easier
-//!
-//! There's a builder for each of the above array types:
-//!
-//!
-//! - [`PointBuilder`]
-//! - [`LineStringBuilder`]
-//! - [`PolygonBuilder`]
-//! - [`MultiPointBuilder`]
-//! - [`MultiLineStringBuilder`]
-//! - [`MultiPolygonBuilder`]
-//! - [`GeometryBuilder`]
-//! - [`GeometryCollectionBuilder`]
-//! - [`RectBuilder`]
-//!
-//! Once you've finished adding geometries to a builder, it's `O(1)` to convert a builder to an
-//! array, by calling `finish()`.
-//!
-//! ## Capacity Counters
-//!
-//! Underlying the builders are growable `Vec`s. E.g. you can think of a `PointBuilder` as a buffer of `x` coordinates and a buffer of `y` coordinates.
-//!
-//! The fastest and most memory-efficient way to construct an array from a set of known geometries
-//! is to make a first pass over these geometries to count exactly how big each part of the Arrow
-//! array must be, allocate _once_ for exactly what you need, and then fill those buffers in a
-//! second pass.
+//! The concrete array definitions.
 //!
+//! All arrays implement the core [GeoArrowArray][crate::GeoArrowArray] trait.
 
 mod coord;
 mod geometry;
@@ -80,3 +29,33 @@ pub use polygon::PolygonArray;
 pub use rect::RectArray;
 pub use wkb::WKBArray;
 pub use wkt::WKTArray;
+
+use std::sync::Arc;
+
+use arrow_array::Array;
+use arrow_schema::Field;
+
+use crate::error::Result;
+use crate::{GeoArrowArray, GeoArrowType};
+
+/// Construct a new [GeoArrowArray] from an Arrow [Array] and [Field].
+pub fn from_arrow_array(array: &dyn Array, field: &Field) -> Result<Arc<dyn GeoArrowArray>> {
+    use GeoArrowType::*;
+
+    let result: Arc<dyn GeoArrowArray> = match GeoArrowType::try_from(field)? {
+        Point(_) => Arc::new(PointArray::try_from((array, field))?),
+        LineString(_) => Arc::new(LineStringArray::try_from((array, field))?),
+        Polygon(_) => Arc::new(PolygonArray::try_from((array, field))?),
+        MultiPoint(_) => Arc::new(MultiPointArray::try_from((array, field))?),
+        MultiLineString(_) => Arc::new(MultiLineStringArray::try_from((array, field))?),
+        MultiPolygon(_) => Arc::new(MultiPolygonArray::try_from((array, field))?),
+        GeometryCollection(_) => Arc::new(GeometryCollectionArray::try_from((array, field))?),
+        Rect(_) => Arc::new(RectArray::try_from((array, field))?),
+        Geometry(_) => Arc::new(GeometryArray::try_from((array, field))?),
+        WKB(_) => Arc::new(WKBArray::<i32>::try_from((array, field))?),
+        LargeWKB(_) => Arc::new(WKBArray::<i64>::try_from((array, field))?),
+        WKT(_) => Arc::new(WKTArray::<i32>::try_from((array, field))?),
+        LargeWKT(_) => Arc::new(WKTArray::<i64>::try_from((array, field))?),
+    };
+    Ok(result)
+}