core: add CustomVectorSource for binary tile data delivery#4377
core: add CustomVectorSource for binary tile data delivery#4377Vladyslav-Odobesku-Intellias wants to merge 8 commits into
Conversation
|
Thoughts @TimSylvester @alexcristici @adrian-cojocaru ? |
Bloaty Results 🐋Compared to main Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-4377-compared-to-main.txtCompared to d387090 (legacy) Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-4377-compared-to-legacy.txt |
|
Benchmark Results ⚡ Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/benchmark-results/pr-4377-compared-to-main.txt |
|
@louwers I've removed mentions of GRPC.
|
|
We will discuss this during the Technical Steering Comittee Meeting tomorrow, 7:30PM CEST. Do you want to join? If so, please e-mail team@maplibre.org and I will send you an invite. |
Introduces a new vector tile source type that accepts pre-encoded MVT tile data via a Kotlin coroutine-based provider interface, bypassing the internal HTTP server bridge. C++ core: CustomVectorSource, CustomVectorTileLoader (actor-based), CustomVectorTile (GeometryTile subclass), RenderCustomVectorSource. Android JNI bridge: fetchTile/cancelTile callbacks via AttachEnv, binary data transfer via GetArrayRegion. Kotlin API: CustomVectorTileProvider (suspend interface), TileData (sealed class), CustomVectorSource (coroutine-per-tile with ConcurrentHashMap-based cancellation). Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Generates diagonal cross patterns per tile as MVT data using a hand- rolled protobuf encoder. Toggle the source on/off with the FAB. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
- setTileError now erases tileCallbackMap + dataCache so errored tiles can be retried; previously the stale entry blocked re-fetches permanently - cancelTile now erases tileCallbackMap so panning back to a cancelled tile correctly re-issues the fetch - CustomVectorTile::setTileError delegates to GeometryTile::setError so loaded=true is set; previously TilePyramid::isLoaded() would stall permanently after any tile error - fetchTile uses CoroutineStart.LAZY + stores job before start to close the race where onRemovedFromMap could miss an in-flight coroutine; finally block uses remove(key, value) to avoid evicting a newer job - setTileData treats empty byte arrays as null (no data) instead of passing a 0-byte buffer to the MVT parser - Add default: assert branch to TileDataFormat switch
Co-Authored-By: Claude <noreply@anthropic.com>
06837e5 to
8352845
Compare
for more information, see https://pre-commit.ci
|
Hi @Vladyslav-Odobesku-Intellias could you merge in main? That should resolve the CI issue. Also there is a merge conflict. |
# Conflicts: # src/mbgl/renderer/render_source.cpp
f2189a3 to
a28a25e
Compare
Bloaty Results (iOS) 🐋Compared to main Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results-ios/pr-4377-compared-to-main.txt |
done, all green |
Introduces CustomVectorSource — a new source type that lets applications deliver pre-encoded binary tile data (MVT) directly to the rendering pipeline, bypassing the GeoJSON encode/decode round-trip of CustomGeometrySource.
Design Proposal
Included in this PR: design-proposals/2025-07-05-custom-vector-source.md
Motivation
Existing options for programmatic tile delivery have limitations:
CustomVectorSource fills the gap for applications that generate tiles on-device, decode from proprietary archives, or stream over non-HTTP transports.
Here is the description of what problem we are solving by introducing such data source. Newer Google Maps SDK provides tile data through GRPC service. That service is launched along the app as a separate process and simply serves tiles through GRPC in its` own format that resembles vector tiles. We cannot request it to service tiles through GRPC + in the vector tiles format. So we need to adapt it. Proposed interface declares a simple interface for the tile data provider to implement.
It everything. Only says that this function is suspendable (coroutine) and under the hood it can be anything - some exotic HTTP tile service or as in our case some GRPC service. For us to now adapt such GRPC service to MapLibre is just about implementing CustomVectorTileProvider by doing a suspendable call to GRPC, which will naturally be suspended and then transform data from one binary format to vector tiles.
Check it yourself
This change also adds one entry to the test scenarios of the android demo app. It simply injects trivial CustomGeometrySource that provides cross tile data (2 lines that form a cross) and sets the style of those lines to be red.
AI Disclosure
AI assistance (Claude) was used for drafting the design proposal document and PR description. All code and architectural decisions are human-authored.