Skip to content

Commit 274bcc6

Browse files
Prohibit DuckDB spatial ops; require H3 hash joins for all geographic filtering (#49)
* h3-guide: add resolution direction and pre-computed parent column join pattern Fixes #44 Two gaps caused a model to spiral for 26 turns on a cross-resolution join: 1. No explicit statement that higher H3 resolution numbers are finer/children 2. No guidance to use pre-computed parent columns (e.g. h6 on an h8 dataset) before falling back to h3_cell_to_parent() * Prohibit all DuckDB spatial ops; require H3 hash joins (Fixes #47) ST_Within/ST_Intersects/ST_Distance and all spatial predicates are always too slow for global datasets — that's the entire reason the workflow pre-computes H3 indices. Add a prominent section 0 making this explicit and directing agents to H3-indexed boundary datasets. * Strengthen spatial ops prohibition in h3-guide; remove duplicate in query-optimization (Fixes #47) h3-guide already covered this — strengthen the existing line to be explicit and non-negotiable, and add the "say so if no H3 boundary" rule. Remove the redundant verbose section added to query-optimization.md. * Refine spatial ops guidance: hex-first, missing hex = pipeline gap, single-geom ops ok --------- Co-authored-by: Carl Boettiger <cboettig@berkeley.edu>
1 parent 1e8fc94 commit 274bcc6

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

h3-guide.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,12 @@
22

33
**Most datasets have H3 hex versions.** Always use them for spatial operations instead of GeoParquet geometry columns.
44

5-
**H3 hex joins replace geometry functions.** Do NOT use `ST_Intersects`, `ST_Contains`, `ST_Area`, or `ST_Within` — these require scanning full polygon geometries and are orders of magnitude slower. Instead, join datasets on their shared H3 index (`h8`, `h0`) to compute overlaps, areas, and containment. If two datasets use different H3 resolutions, convert with `h3_cell_to_parent()`.
5+
**Always use H3 hex datasets for filtering and joining — never spatial predicates on GeoParquet.**
6+
When a dataset appears in the STAC catalog as GeoParquet, a hex-indexed version almost always exists alongside it. Find and use the hex version. Never use `ST_Within`, `ST_Intersects`, `ST_Contains`, or similar predicates to filter or join large datasets — on global data these run 10+ minutes and return nothing useful.
7+
8+
If you browse the catalog and only find a GeoParquet with no hex equivalent, **say so** rather than falling back to spatial predicates. A missing hex version is a data pipeline gap (not something to work around silently).
9+
10+
Small spatial operations on a **single already-retrieved geometry** are fine — e.g. `ST_Centroid` or `ST_AsText` on one result row for display or map zoom. This is different from using spatial predicates to scan or filter a dataset.
611

712
## Resolution Direction
813

0 commit comments

Comments
 (0)