fix(catalog): read per-asset table:columns in extractColumns (#311)

[cboettig]

Closes #311.

Problem

extractColumns (app/dataset-catalog.js) — the client-side map's schema channel feeding tooltips, legends, layer columns, and isParentContainer detection — read only the collection-level collection['table:columns']. Meanwhile the LLM's schema channel (get_schema → MCP get_stac_details) already reads per-asset table:columns.

The catalog/publisher standard is moving to per-asset table:columns (data-workflows verify-stac.py hard-fails collection-level). Once publishers drop the collection-level block, extractColumns would return [] → map tooltips/legends lose column info, and isParentContainer (keyed on ds.columns.length === 0) would misfire on real leaf datasets.

Change

Repoint extractColumns at the per-asset table:columns on the queryable parquet assets:

• Union columns across the queryable parquet assets (hex + flat GeoParquet).
• Prefer the H3 hex asset (the primary query target per the MCP's query-optimization guidance) so its richer, H3-aware column definitions win on name collisions; the flat GeoParquet fills any gaps.
• Drop geometry (geom/geometry) columns; dedup by name.
• Non-parquet assets (e.g. pmtiles) are ignored when a parquet asset is present.
• Fallback: when no per-asset schema is present, fall back to the legacy collection-level table:columns block, so catalogs mid-migration keep working.

The map UI now reads the same canonical location as the LLM renderer, and the collection-level block can be retired catalog-wide once mcp-data-server#303 lands (render-time dedup) with no consumer left behind.

Notes (out of scope, from the issue)

• TOOL_RESULT_CAP truncation of large multi-asset get_schema results is addressed by mcp-data-server#303's dedup; no change here.
• The two independent startup STAC caches (geo-agent boot vs MCP pre-warm) remain a known consistency footgun; not addressed here.
• Per the issue, only geometry is dropped — H3 index columns (h0/h8/h9/h10) are retained.

Testing

npm test — 492 passing. Added 4 extractColumns cases: per-asset union with hex precedence, pmtiles ignored when parquet exists, and legacy collection-level fallback.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	56.9%	5004 / 8794
🔵	Statements	56.87%	5004 / 8798
🔵	Functions	63.73%	181 / 284
🔵	Branches	83.57%	1099 / 1315

File Coverage

File	Stmts	Branches	Functions	Lines	Uncovered Lines
Changed Files
app/dataset-catalog.js	91.56%	80.61%	100%	91.56%	87-89, 115-117, 142-151, 171-173, 183-184, 433-438, 467-508, 592, 871-877, 890-895

Generated in workflow #165 for commit 0355ae3 by the Vitest Coverage Report Action

[cboettig]

Regression check (requested: headless / gold-question performance, dse-nimbus qwen)

Rather than only sampling the LLM, I verified this against the live STAC at ground truth, because in headless mode extractColumns reaches the model through exactly one channel: generatePromptCatalog() uses ds.columns only for isParentContainer detection (column schemas are never front-loaded — the LLM gets them via get_schema, which forwards raw STAC and is untouched here). So the question "can qwen's answers regress?" reduces to "does the assembled system prompt change?"

Loaded all 6 golden-bench app configs (biodiversity, bosl-high-seas, ca-30x30, global-30x30, tpl-ca, wetlands-v2 — 133 dataset instances) against the production catalog, on main vs this branch:

metric	result
datasets that lost any column vs main	0
datasets fixed (main returned [] → now populated)	108
isParentContainer flips	0
assembled prompt-catalog text (content, order-normalized)	byte-identical on all 6 apps

Gold-question / qwen verdict: the system prompt qwen receives is content-identical, so no gold-question regression is possible from this change — a live qwen matrix run can only reconfirm this while adding sampling noise. Happy to run the k8s matrix bench if you want the empirical datapoint anyway.

What the check surfaced (and why the PR changed)

The 108 fixed datasets are the important number: most collections have already migrated to per-asset-only table:columns (e.g. wdpa has no collection-level block at all), so main's extractColumns was already returning [] for them — the map UI already had no column info. That's #311's predicted breakage, already live.

The first commit (per-asset only, per the issue's literal "hex + flat GeoParquet") introduced its own regression: wwf-ecoregions-2017 and overture-divisions-* carry no per-asset block on the flat parquet asset and only join keys (_cng_fid, h8, h0) on the hex asset — their display columns still live only in the collection-level block (20–26 → 3). The catalog is mid-migration (data-workflows #404/#369 backfill incomplete).

Fix: union the per-asset blocks (canonical, hex-preferred) with the collection-level block as a lower-priority trailer — which is what the MCP get_stac_details renderer already does ("per-asset + collection-level trailer"). This fixes the already-migrated datasets and keeps the not-yet-backfilled ones whole, and when the collection block is finally retired the per-asset union already carries everything (no consumer left behind — the issue's stated goal).