task/WG-321: improve how vector files are imported - #294
Merged
nathanfranklin merged 14 commits intoJul 30, 2026
Conversation
- Floor tippecanoe's guessed max zoom so sparse/global vector data still produces renderable tiles (and simplify geojson_to_pmtiles to its used args) - Include vector-asset features in the no_asset_vector feature query - List the containing directory when importing so only existing companion shapefile files are fetched (avoids noisy 404s) - Serve .pmtiles assets with byte-range support and CORS preflight in nginx - Add a decode-and-inspect test plus a point+polygon fixture
tippecanoe drops features at low zoom by default, leaving a scattered point layer nearly empty at the world view. Add --drop-rate=1 so every feature is kept at every zoom (--drop-densest-as-needed still caps oversized tiles), plus a worker test asserting >900 of the 1000 points survive at each zoom.
1 task
nathanfranklin
requested review from
rstijerina,
shayanaijaz,
sophia-massie and
taoteg
July 24, 2026 16:28
sophia-massie
approved these changes
Jul 29, 2026
sophia-massie
left a comment
Contributor
There was a problem hiding this comment.
Nathan and I huddled - found with geoJSON with no metadata, the message "Click a feature on the map to see its attributes." could be confusing because its attributes will not appear after clicking.
Also noticed that the mouse isn't detecting that the pmtiles are clickable even though they are and work correctly i.e. change metadata depending on the point selected.
nathanfranklin
deleted the
task/WG-321-fix-importing-for-geojson-shapefiles
branch
July 30, 2026 18:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview:
This PR along with Hazmapper PR migrates vector file ingest from "N features in PostGIS" to a "1 Feature + PMTiles asset" model.
Uploaded vector files (e.g. GeoJSON, shapefile) are reprojected to EPSG:4326, then the geojson is run through tippecanoe to produce a static
.pmtilesarchive, and stored as a singleFeatureAsset(asset_type="vector") on a Feature. The pmtile is served directly to the client (over HTTP range requests).Note: The GeoJSON-body endpoint for adding individual features (
addGeoJSON) is unchanged.Related Jira tickets:
Summary of Changes:
TippecanoeService(geoapi/services/tippecanoe.py): wraps tippecanoe as a subprocess to build a.pmtilesarchive from GeoJSON. Uses-zgto guess the max zoom, floored via--smallest-maximum-zoom-guessso sparse/large-extent data (which otherwise guesses zoom 0) still produces adequately-precise tiles. (Related reading is https://felt.com/blog/tippecanoe-display-scale-precision and https://medium.com/fika-blog/the-dark-art-of-vector-map-tiling-b417a3813df5)VectorService.convert_to_geojson(geoapi/services/vectors.py): reads any supported vector format via geopandas, reprojects to EPSG:4326, strips Z coordinates, extracts the bounding box, and writes GeoJSON to a temp dir. DefinesSUPPORTED_VECTOR_EXTENSIONS; removes the now-orphanedprocess_shapefile.FeaturesService.fromVectorFile? converts to tippecanoe and then copies the.pmtilesinto the project asset dir.geoapi/tasks/external_data.py): the single-file Tapis import now lists the containing directory so only companion shapefile files that actually exist are fetched (avoids noisy 404s) ad non-required-file log downgraded to warning.Testing Steps:
devops/Dockerfile.worker):docker compose build workers(or your local build target).Notes:
.gpkg) — a multi-layer container (and can hold raster tiles), effectively "the 5 shapefiles users used to send, bundled." Our "1 file = 1 Feature" model would silently ingest only its first layer. Proper support means exploding its vector layers into N Features and surfacing any embedded rasters as internal tile layers. So this could be done laterTODO