Skip to content

Conversation

@davenquinn
Copy link
Member

@davenquinn davenquinn commented May 22, 2025

Integrate tileserver from the UW-Macrostrat/tileserver repository, which is now archived.

We're making this integration because the tileserver needs to become multiple services anyway, due to different technical requirements between Mapnik, RasterIO, and other dependencies for different types of tile serving. We'd also like to more closely integrate tile services with other "API v3" elements in the future. Housing all of these components in the same codebase should be helpful for integration tests, sharing components between services, etc. However, this will be slightly unwieldy (especially with a lot of noise in github issues). We'll see how it goes.

Changes

  • Splice commits from tileserver repository into this one. The join point is at this commit.
  • Split tileserver service into "legacy" and "modern" services with different Python/GIS stacks
  • Moved cache creation/management logic into a separate macrostrat.tileserver_utils module
  • Start the process of removing TimVT
  • Create a new prototype image compilation pipeline
    1. Run scripts/prepare-services to compile wheels of local Python packages, stage them in service directories, and lock dependencies
    2. Build docker images
  • Add a scripts/tag-releases script to create version tags that can be consumed by Github CI pipelines

Future updates

  • The Docker image compilation pipeline can be streamlined and augmented as problems come up. In particular, a well-documented release workflow will be helpful.
  • Create a shared GitHub CI workflow for images
  • Improve image building and CI for API v3

These next steps are tracked in #150

Preparing Docker images using Poetry was quite intricate during this, suggesting that migrating to UV (#89) might be worthwhile

davenquinn added 30 commits July 5, 2022 12:16
* multiscale-layers:
  Both carto and carto_slim seem to work now
  Added a layer definition for carto-slim
  Basic layer defs for carto
  A pretty nice dynamic tile layer
  Updated basic function layer
  A basic function layer
  A halfway reasonable tileserver API
  Basic function layer
  Updated management SQL functions
  Created a CLI for producing SQL functions
  Added tile-bounds calculating fixtures
  Moved reference queries to same file
  Added outdated seed script files from macrostrat-utils as reference material
* multiscale-layers:
  Handle cors at the webserver level
  Updated some functions
  Updated tile utils
* v2:
  Handle cors at the webserver level
  Updated some functions
  Updated tile utils
  Both carto and carto_slim seem to work now
  Added a layer definition for carto-slim
  Basic layer defs for carto
  A pretty nice dynamic tile layer
  Updated basic function layer
  A basic function layer
  A halfway reasonable tileserver API
  Basic function layer
  Updated management SQL functions
  Created a CLI for producing SQL functions
  Added tile-bounds calculating fixtures
  Moved reference queries to same file
  Added outdated seed script files from macrostrat-utils as reference material
* cartocss:
  Better checking for bad colors
  Simplify stylesheet generation
  Greatly increase the speed of Mapnik tile generation by using data-driven styles
  Updated map display
  Mapnik layers work amazingly well
* mapnik:
  Slightly nicer style management
  Better checking for bad colors
  Simplify stylesheet generation
  Greatly increase the speed of Mapnik tile generation by using data-driven styles
  Updated map display
  Mapnik layers work amazingly well
  Basic image serving works
  Added some extremely basic tests
  Added skeletal build scripts for tileserver docs
  WIP update tile pages
  Remove some build dependencies
  Add mapnik compilation to dockerfile
* tile-caching:
  Updated URLs for image tiles
  Updated cache definitions
  Updated cache management
  Saves tiles to cache but doesn't recover them properly
  Start working towards a viable cache system
davenquinn and others added 24 commits May 21, 2025 04:57
* separate-legacy-tileserver:
  Updated release tagging scripts
  Updated lockfiles and dependencies
  Updated services
  Allow mvt to be served from legacy tile server as well
  Works better with tile caching
  Streamlined tileserver utils
  Improved scripts to build images
  Slightly improved cache pool
  Slightly adjust styles for efficiency
  Updated map styling
  Updated map cache queue
  Tileserver actually works
  Partially working legacy tileserver
  Starting point for separating tile servers
* update-tileserver:
  Draft new Github actions workflows
  Small adjustments to index page
  Streamlined tileserver and removed archival files
  Updated legacy tileserver and docker files
  Start streamlining tileserver code
  Updated tileserver
  Updated macrostrat.tileserver
  Updated tileserver building
  Include new utility functions
…rate-tileserver

* origin/integrate-tileserver:
  Format code and sort imports
@davenquinn davenquinn merged commit 75393fa into main May 22, 2025
3 of 5 checks passed
@davenquinn davenquinn deleted the integrate-tileserver branch May 23, 2025 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants