task/WG-703: create and provide nightly public archive (pmtiles + geojson) for published DS maps - #300
Open
nathanfranklin wants to merge 26 commits into
Open
task/WG-703: create and provide nightly public archive (pmtiles + geojson) for published DS maps#300nathanfranklin wants to merge 26 commits into
nathanfranklin wants to merge 26 commits into
Conversation
- Floor tippecanoe's guessed max zoom so sparse/global vector data still produces renderable tiles (and simplify geojson_to_pmtiles to its used args) - Include vector-asset features in the no_asset_vector feature query - List the containing directory when importing so only existing companion shapefile files are fetched (avoids noisy 404s) - Serve .pmtiles assets with byte-range support and CORS preflight in nginx - Add a decode-and-inspect test plus a point+polygon fixture
tippecanoe drops features at low zoom by default, leaving a scattered point layer nearly empty at the world view. Add --drop-rate=1 so every feature is kept at every zoom (--drop-densest-as-needed still caps oversized tiles), plus a worker test asserting >900 of the 1000 points survive at each zoom.
nathanfranklin
requested review from
rstijerina,
shayanaijaz,
sophia-massie and
taoteg
July 29, 2026 01:48
…task/WG-703-public-designsafe-pmtiles
Base automatically changed from
task/WG-321-fix-importing-for-geojson-shapefiles
to
main
July 30, 2026 18:41
nathanfranklin
changed the title
nathanfranklin
changed the title
nathanfranklin
commented
Jul 31, 2026
Collaborator
Author
There was a problem hiding this comment.
need to allow public in auth check or have different location like in Core-Portal-Deployments PR
Collaborator
Author
There was a problem hiding this comment.
could just handle in /auth-check and then add http read support for pmtiles in this block
Collaborator
Author
There was a problem hiding this comment.
didn't handle in /auth-check as lots of requests would be passed on to our geoapi backend. so just adding a location block. a919dc9
Conflicts fixed in: geoapi/services/tippecanoe.py geoapi/tests/api_tests/test_vectors_service.py geoapi/tests/services/test_tippecanoe_service.py
Collaborator
Author
|
Also need to provide one workaround for a large raster in map associated with https://www.designsafe-ci.org/data/browser/public/designsafe.storage.published/PRJ-6361/ |
Collaborator
Author
fixed with cae117c |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview:
This PR provides a pmtile containing all public maps associated with published DesignSafe projects.
Nightly, generate a single combined PMTiles archive of all public Hazmapper map data belonging to published DesignSafe projects for this deployment*, write it to a shared public asset area served by nginx, and expose an unauthenticated route that returns a manifest (archive URL + metadata). Consumer is WG-671 (Recon Portal).
Live example viewers (Leaflet + MapLibre), showing an example archive with clickable popups that link back to Hazmapper and DesignSafe:
If you want the visual, add a clickable thumbnail under those links:
Related Jira tickets:
Testing Steps:
then
curl http://localhost:8000/designsafe/published-maps/pmtiles.3. Evaluating pmtiles from prod deployment: see https://nathanfranklin.github.io/published-ds-maps-examples/
Notes