Skip to content

streams: put NHDPlus HR on the map, stop styling base NHD on Klamath-only STREAMORDER - #116

Open
cboettig wants to merge 2 commits into
mainfrom
fix/streams-followups-111
Open

streams: put NHDPlus HR on the map, stop styling base NHD on Klamath-only STREAMORDER#116
cboettig wants to merge 2 commits into
mainfrom
fix/streams-followups-111

Conversation

@cboettig

@cboettig cboettig commented Aug 8, 2026

Copy link
Copy Markdown
Member

Closes the two map-layer loose ends #111 deferred until data-workflows#205 landed, and refreshes the AGENTS.md row that still described the problem as open.

The map was still styled on the broken field

layers-input.json drew "Streams by Order (NHD)" with line-width interpolated on ["get","STREAMORDER"] and exposed STREAMORDER on hover. That attribute is populated on 8.7% of California's flowlines, 97.8% of it HUC4 1801 — so outside the Klamath the width expression had no input and the tooltip field read empty. #111 flagged it and deliberately deferred rather than paper over it.

layer before after
usgs-nhdplus-hr-flowline absent "Stream Order (NHDPlus HR)" — width + tooltip on streamorde
usgs-nhd-streams-by-order "Streams by Order (NHD)", width/tooltip on STREAMORDER "Stream Extent (NHD)" — constant width, no order field

The split matches the routing rule already in system-prompt.md: order and network attributes from NHDPlus HR, extent and flow permanence from base NHD. Display names now say which layer is which so the toggle list does not offer two things called "by order".

Asset ids and field names verified against the published PMTiles, not the parquet schema:

$ curl -s .../nhdplus-hr/flowline/stac-collection.json
id: usgs-nhdplus-hr-flowline
  asset: flowline-pmtiles | vector:layers: ['flowline']

# PMTiles metadata, vector_layers[0]
layer: flowline | minzoom 0 | maxzoom 10
fields: [..., 'ftype', 'gnis_name', 'lengthkm', 'streamorde', 'vpu_unit', ...]

Tiles are display-only — tippecanoe dropped features on dense tiles, so no quantity should ever be computed from them. That is a data-layer property and is documented in the collection.

AGENTS.md row was stale

The streams row read "App rule pending; upstream in progress", cited "99.65% coverage across all 13 CA basins", and said NHDPlus HR "is being added alongside". All three upstream issues are closed, coverage is 100.00% across 266 national units, and the rule shipped in #112/#114.

More importantly it did not carry the lesson that actually cost a gate run: the collection is national and clipped to nothing, so an unmasked California query reads 924,634 km against 641,393 km in-state and 19.0% conserved against 27.5% — the same mechanism as the pinyon-juniper row directly above it. The rewritten row leads with the routing rule and the mask, keeps the four ruled-out investigations (now including the false "H3 can only measure area" premise), and keeps the standing expectation that our lengths run ~3x the partner's forever.

Gate

regression + commentary on z-ai/glm-5.2 + deepseek/deepseek-v4-flash-0731. Adding a collection to layers-input.json changes what the agent sees at prompt-assembly time, so this is gated rather than shipped on inspection even though no SQL guidance changed.

Refs #111

…only STREAMORDER

The map half of #111 was deferred until data-workflows#205 landed. It has.

- Add usgs-nhdplus-hr-flowline (flowline-pmtiles) as "Stream Order (NHDPlus HR)",
  width and tooltip on `streamorde` — populated on 100% of in-network non-coastline
  length, so the width expression now has an input everywhere.
- The base NHD layer keeps extent only: rename to "Stream Extent (NHD)", drop the
  STREAMORDER width interpolation and the STREAMORDER tooltip field. That field is
  empty outside HUC4 1801, so 91% of California's flowlines rendered at the
  expression's fallback width and hovered an empty attribute.
- AGENTS.md: the streams row said "App rule pending; upstream in progress" with a
  13-basin 99.65% figure. Upstream is closed, the rule shipped in #112/#114, and the
  row now carries the lesson that cost the run — the collection is national and
  unclipped, so mask to California.

Refs #111
@cboettig

cboettig commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

Gate: green

regression + commentary, z-ai/glm-5.2 + deepseek/deepseek-v4-flash-0731, 2 trials, prod MCP, APP_BRANCH=fix/streams-followups-111. Jobs hmx-ca-30x30-bench-pr116-{regression,commentary}-20260808-2342. Graded and judged with collect_run.py + judge.py; the pre-change natl2 run on main was collected and judged identically so the comparison is like-for-like rather than against a remembered number.

this branch main (natl2)
regression accuracy, glm 16 pass / 2 fail 15 pass / 1 close / 2 fail
regression accuracy, deepseek 16 pass / 1 close / 1 fail 15 pass / 1 close / 1 fail / 1 judge-error
numeric auto-grade 6/6 both models 6/6 glm, 5/6 deepseek
commentary accuracy 16/16 glm, 15/16 deepseek 16/16 both

car-28-headwater-streams, the cell that catches an unclipped or mis-weighted stream answer: 4/4 pass at 27.46–27.5 against gold 27.0, identical to main. Every cell routed to usgs-nhdplus-hr-flowline, filtered innetwork = 1 AND streamorde IN (1,2), masked on h8+h0, and length-weighted.

No timeouts on either tier.

Compliance axis moved in both directions and neither is this branch

Judged compliance came out 14/18 (regression) and 11/16 (glm commentary) here against 16–17/18 and 10/16 on main. Reading the individual verdicts, the deltas are judge artifacts, not behaviour — I filed geo-agent-benchmark#20 with the evidence:

  • E:fabricated-technical-detail fires on text quoted verbatim from the STAC description the model is given ("30 m statewide vegetation raster compiling best-available land-cover (~1990–2022)", "45,811 polygons"). Both are checkable in the published collection JSON.
  • D:invented-per-feature-target fires on answers stating the opposite — "the 30% goal is statewide and is not a benchmark for any individual habitat" — and on statewide goal arithmetic that is car-05's own gold.

The same cells flip verdicts between two judgings of unchanged answers, so I would not read a 2-cell compliance move on this tier as signal in either direction.

Not addressed here, deliberately

The one real defect the gate exposed is that a model can route and mask correctly and still weight the conserved fraction by cell area instead of stream length — 25.47 vs 27.46 on identical input, reproduced exactly. That is generic SQL method, not app interpretation, so per AGENTS.md's layer table it went to mcp-data-server#363 rather than into system-prompt.md.

@cboettig

cboettig commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

Pushed d517543 after the gate: AGENTS.md prose only, trimmed to state the rule rather than narrate the fix (per the standing docs convention). AGENTS.md is not part of the deployed app — it is not read at prompt-assembly time — so this cannot move behaviour and does not invalidate the run above, which tested bd070fc. The app-facing diff (layers-input.json) is unchanged from the gated commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant