Skip to content

Proximity / "nearest X" queries — engine choice and the ST_-steering tension #250

Description

@cboettig

Motivation

Now that the agent can get the user's location (get_user_location, #244/#249) and resolve named places (geocode), a natural next class of question opens up:

"What's my nearest national park?" · "nearest protected area" · "parks within 5 km of here" · "how far am I from the coast?"

These are proximity / distance questions. They're a poor fit for our H3 hex view, which is built for aggregation (counts/sums per cell) and overlap (hex-ID joins), not nearest-neighbor or true distance. This issue is to decide where these queries should be answered and how to expose that to the agent — without regressing the H3 discipline we've deliberately enforced.

Why hexes don't naturally answer this

"Nearest feature" is a point-geometry operation, orthogonal to hex aggregation. We do have H3 primitives that could approximate it:

  • k-ring expansionh3_grid_disk(...) is already used (see mcp-data-server/query-optimization.md) to build neighbour cell sets.
  • H3 distance — the H3 DuckDB extension exposes great-circle / grid-distance functions.

But per production logs (Carl's recollection — worth re-confirming from proxy logs) these are very inefficient in practice — e.g. distance from one point to all candidate cells is a full scan, and ring-expansion-until-hit is awkward to express in SQL.

The candidate engines

Engine Fit for "nearest over the full dataset" Notes
DuckDB ST_* (MCP) Best — has the complete geometry, true geodesic distance/buffer But see the steering tension below
H3 k-ring / H3 distance (MCP) Approximate; resolution-limited Functions exist but are inefficient per logs
Client-side Turf.js ❌ for tiled data Browser only holds rendered PMTiles features → would silently answer "nearest visible park". Correctness trap. Fine only for GeoJSON / small / the drawn region.

Conclusion from the design chat: for "nearest over a national dataset," the data-completeness argument puts this server-side. Turf is the wrong vehicle for this (more below).

The real tension: we've steered models away from ST_

The MCP tool docs deliberately discourage spatial predicates because local models over-reach for them and write ST_ joins where hex math is correct and ~1000× faster. From mcp-data-server/h3-guide.md:

"Never use ST_Within, ST_Intersects, ST_Contains, or similar predicates to filter or join large datasets — on global data these run 10+ minutes and return nothing useful." (line 6)

"All spatial operations are hex joins... Never use ST_Within, ST_Intersects, ST_Centroid, or any spatial function." (line 18)

So naively telling the agent "use ST_Distance for nearest" risks undoing that discipline — models would generalize it back to aggregations/joins where H3 wins. Until we can robustly test that models navigate the H3-vs-ST_ choice accurately, the safe default is to keep ST_ heavily fenced.

The precedent that resolves it: hex-narrow → ST_-refine

h3-guide.md already carves one narrow ST_ exception — Problem 4 (exact line mileage inside an AOI). Its shape is the template for distance too:

hex SEMI JOIN to a small candidate set first, then ST_Intersection on the GeoParquet geometries joined by the per-feature key (_cng_fid).

A "nearest" query can follow the identical discipline:

  1. k-ring from the user's H3 cell outward to gather a small candidate set of nearby features (bounded, hex-only).
  2. ST_Distance on just those candidates' source geometry → exact nearest, ORDER BY ... LIMIT 1.

This keeps ST_ off the global table (the actual banned thing) while giving an exact answer on a handful of rows. It's a new narrow exception in the same spirit as Problem 4, not a relaxation of the global rule.

Key unknowns to resolve first

  1. Do the relevant parquets carry per-feature geometry (a geometry/WKB column), or only H3 + attributes? Problem 4 joins to a separate GeoParquet for ST_Intersection — so the geometry likely lives in a sibling GeoParquet, not the hex file. Confirm for PAD-US / parks before designing.
  2. How bad are the H3 distance functions really? Re-confirm the "inefficient" claim from proxy logs — if a bounded k-ring makes them acceptable, an all-H3 path avoids ST_ entirely.
  3. Can the agent drive the hex-narrow→ST_-refine pattern reliably, or does it need a dedicated tool / documented recipe to stay on rails?

On Turf.js (raised in discussion)

Not overreach as a concept, but not the vehicle for "nearest" — the visible-only correctness trap above. Its genuine, non-overlapping niche is client-native geometry that never round-trips to MCP: ops on the drawn region (get_drawn_region) and small GeoJSON layers — buffer/area/along/distance-to-clicked-point. Adopting it there is reasonable when those use-cases arrive; adopting it for nearest would mean a second geometry engine (drift/consistency cost) plus tool-surface growth (#225). Decide Turf on its own merits, separately from this.

Possible paths forward (not deciding here)

  • A. Documented recipe only. Add a "Problem N — proximity / nearest" section to h3-guide.md with the k-ring→ST_Distance pattern. No new tool. Cheapest; leans on the agent following the recipe.
  • B. MCP-side capability. A nearest_features / distance_to tool in mcp-data-server that encapsulates the pattern server-side, so the model can't mis-shape it. Keeps spatial logic where the full data lives.
  • C. Eval first. Build a small eval of H3-vs-ST_ decisions before relaxing any steering; only then expose proximity broadly.
  • D. Defer. Wait for real demand; revisit when a deployed app actually needs it.

Out of scope / related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions