Experimental image: raster + zarr DuckDB extensions on a separate tag and endpoint - #358
Merged
Conversation
…dpoint
Adds a second image (Dockerfile.experimental -> :experimental) layered on the
default one, carrying the duckdb raster and zarr community extensions, plus a
testing-only endpoint at experimental-duckdb-mcp.nrp-nautilus.io.
raster gives RT_ReadCells/RT_Read (GDAL) so a small GeoTIFF can be joined to
catalog GeoParquet directly, without a hex ingest first; zarr gives read_zarr.
Both stay off the default endpoint: raster pulls GDAL into the query path, and
RT_ReadCells materializes one row per pixel, which is only safe for small
rasters.
One codebase serves both tags. server.py LOADs whatever EXTRA_DUCKDB_EXTENSIONS
names per connection (unset on the default image, so behaviour is unchanged) and
injects experimental-extensions.md into the query tool description only where the
extras are actually loaded. Extension names are restricted to identifier
characters since LOAD takes no parameters.
The guidance carries the two traps found while testing: RT_ReadCells emits
GEOMETRY('EPSG:4326') while catalog GeoParquet is GEOMETRY('OGC:CRS84'), so the
join needs ST_SetCRS; and unfamiliar rasters need a cols*rows check before read,
with the hex asset as the answer for anything large.
The raster extension vendors a GDAL whose libcurl is built with the RedHat CA path (/etc/pki/tls/certs/ca-bundle.crt), which does not exist on the python:slim Debian base. Every /vsicurl/ read failed with 'CURL error: error setting certificate file' while RT_* functions themselves loaded fine, so the extension looked healthy right up until it touched the network.
The area advice was wrong in the direction that matters. It told models to assign each pixel an H3 cell and sum h3_cell_area(), which charges every pixel a whole hex's area rather than the pixel's own — inflating the test fire by ~29% (110.5 ha against a true 85.9). h3_cell_area is for hex rows, not raster rows. Replaced with the pixel size from the raster's own geotransform, including the element order, since $.transform is [originX, dx, rotX, originY, rotY, dy] and the pixel size is at indices 1 and 5 — indexing 0 and 4 silently yields zero area. Found by running the documented recipe against the Eureka fire on the live endpoint and reconciling with the published R analysis (87.16 ha).
This was referenced Aug 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for #354 (streams: raster/zarr reads), kept off the default image.
Approach
Rather than adding extensions to the production image, this makes the extra-extension set a deployment knob and ships a second image/endpoint that turns it on:
EXTRA_DUCKDB_EXTENSIONS(comma-separated) isLOADed per connection, alongside the existingSETUP_SQLstatements. Unset on the default image, soduckdb-mcpbehaves exactly as before — same extensions, same tool description, same bytes of injected guidance.Dockerfile.experimentalinstalls raster + zarr and sets the variable;docker-experimental.ymlpublishes it under its own tag.k8s/experimental-{deployment,service,ingress}.yamlrun it as a separate endpoint, so an experiment can't take down the production head.experimental-extensions.mdcarries the guidance for the extras and is injected only when the extras are actually loaded, so the default deployment'squerytool description is unchanged.Safety notes
^[A-Za-z0-9_]+$before interpolation —LOADtakes no bind parameters, so this is the guard against injection through the env var.LOADwarns to stderr rather than raising: a missing extension degrades that connection's capability instead of failing every query the replica serves. Same posture as the existingSETUP_SQLloop.Status
Not wired into any production path. The default image and the
duckdb-mcpdeployment are untouched; this only adds files plus the opt-in block inserver.py. Merging it does not change what prod serves.Split out of the branch these commits were already sitting on so they can be reviewed on their own.