Add PMTiles basemaps to iTowns by ymoisan · Pull Request #2662 · iTowns/itowns

ymoisan · 2025-12-19T15:39:17Z

I open this PR as per this issue and this discussion.

As I stated in the discussion item, I would like to be able to use PMTiles instead of OGC web services, provided that all "flavours" of PMTiles (MVT, raster, elevation) cover the requirements for iTowns to work. If so, it is my opinion that functional requirements for iTowns could be dramatically relaxed : no web server needed, just simple http access to files. Simplicity in using different basemaps and other map artifacts than what is currently provided would make iTowns easier to personnalize for other organizations, especially those that do not have the breadth of OGC web services currently needed to accommodate iTowns.

I would like this PR to serve as a starting point for discussing the integration of "cloud-native" file formats in iTowns. The example provided is just to show PMTiles of MVT type can be overlaid in iTowns. My hope is that a well prepared PMTiles could provide the same output than OGC services and therefore could be a substitue rather than mererly an overlay.

ymoisan · 2026-01-06T19:06:07Z

The question of "why PMTiles" was raised in a previous comment. I think a nice answer lies in this article Open Source Mapping for News: Reuters.

airnez · 2026-01-09T09:24:26Z

Hi @ymoisan
Thank you for this interesting contribution :)
I'll start reviewing this PR in the next few days.

airnez

Thank you for this contribution ! PMTiles is indeed a very interesting format.
I tested your branch, and it looked fine to me.
With one data source, it seems that some tiles are broken, do you know why ? I suspect an iTowns-related problem, after reading your code.
https://r2-public.protomaps.com/protomaps-sample-datasets/tilezen.pmtiles

airnez · 2026-01-15T15:39:43Z

packages/Main/src/Source/PMTilesVectorSource.js

+        super(source);
+
+        this.isPMTilesVectorSource = true;
+        this.isVectorSource = true;


Is setting isVectorSource necessary ? Source.js defines

this.isVectorSource = (source.parser || supportedParsers.get(source.format)) != undefined;

I guess this is enough considering that you set

source.format = 'application/x-protobuf;type=mapbox-vector';

What do you think ?

airnez · 2026-01-15T15:47:22Z

packages/Main/src/Source/PMTilesVectorSource.js

+    getDataKey(extent) {
+        if (extent.isTile) {
+            return `z${extent.zoom}r${extent.row}c${extent.col}`;
+        }
+        return `z${extent.zoom}r${extent.row}c${extent.col}`;
+    }


This might not be necessary. It always returns the same value whether extent.isTile is true or false. You could rely on Source.js code that does use the same key scheme.

`z${extent.zoom}r${extent.row}c${extent.col}`

airnez · 2026-01-15T16:17:12Z

packages/Main/src/Source/PMTilesVectorSource.js

+            .then((data) => {
+                if (!data) {
+                    // No tile data - return empty collection
+                    return Promise.resolve(null);


I think it could be safer to return an empty FeatureCollection here.

Promise.resolve(new FeatureCollection(out));

airnez · 2026-01-15T17:11:10Z

examples/source_pmtiles_vector.html

+
+                const opacityToUse = userOpacity;
+
+                pmtilesSource.clearCache();


I don't think that you need to empty the source cache to reload the layer. It seems to work after commenting this line. In Itowns, it seems that clearing the cache is more a View / Layer responsability than source.

airnez · 2026-01-16T08:42:34Z

packages/Main/src/Source/PMTilesVectorSource.js

+    /**
+     * Called when layer is added. Sets up caching.
+     *
+     * @param {Object} options - Options
+     */
+    onLayerAdded(options) {
+        super.onLayerAdded(options);
+
+        // Setup cache for features
+        if (!this._featuresCaches[options.out.crs]) {
+            this._featuresCaches[options.out.crs] = new LRUCache({ max: 500 });
+        }
+    }


I think that Source.js already handles that for you. Calling super will always instanciate features cache for this layer. You could simpy not implement this method, what do you think ?

airnez · 2026-01-16T14:13:54Z

packages/Main/src/Source/PMTilesSource.js

+ * @property {number} zoom.min - The minimum zoom level of the source.
+ * @property {number} zoom.max - The maximum zoom level of the source.
+ */
+class PMTilesSource extends Source {


I understand that this parent source will be useful later to specify a Raster PMTiles Source ?

airnez · 2026-01-16T15:46:45Z

packages/Main/src/Source/PMTilesVectorSource.js

+        this.styles = source.styles || {};
+
+        // Parser for MVT tiles
+        this.parser = VectorTileParser.parse;


I dont think this is needed, as Source.js already sets this parser for application/x-protobuf;type=mapbox-vector WDYT ?

airnez · 2026-01-16T15:58:56Z

packages/Main/src/Source/PMTilesVectorSource.js

+ * @property {boolean} isPMTilesVectorSource - Used to checkout whether this
+ * source is a PMTilesVectorSource. Default is true. You should not change this,
+ * as it is used internally for optimisation.
+ * @property {Object} layers - Object containing layer definitions for styling.


This differs from VectorTilesSource, but I think this makes sense as PMTiles are not advertised through mapbox styles. This directly corresponds to the format expected by the parser

airnez · 2026-01-16T16:15:59Z

packages/Main/src/Source/PMTilesVectorSource.js

+    /**
+     * Set visibility of a specific source layer.
+     *
+     * @param {string} layerName - The name of the source layer
+     * @param {boolean} visible - Whether the layer should be visible
+     */
+    setLayerVisibility(layerName, visible) {
+        if (!this._layerVisibility) {
+            this._layerVisibility = {};
+        }
+        this._layerVisibility[layerName] = visible;
+
+        // Update layers object based on visibility
+        if (this._originalLayers && this._originalLayers[layerName]) {
+            if (visible) {
+                this.layers[layerName] = [...this._originalLayers[layerName]];
+            } else {
+                // Set to empty array so parser skips this layer
+                this.layers[layerName] = [];
+            }
+        }
+    }
+
+    /**
+     * Set visibility of all source layers.
+     *
+     * @param {boolean} visible - Whether all layers should be visible
+     */
+    setAllLayersVisibility(visible) {
+        if (this._originalLayers) {
+            Object.keys(this._originalLayers).forEach((layerName) => {
+                this.setLayerVisibility(layerName, visible);
+            });
+        }
+    }
+
+    /**
+     * Get visibility state of a layer.
+     *
+     * @param {string} layerName - The name of the source layer
+     * @returns {boolean} Whether the layer is visible
+     */
+    getLayerVisibility(layerName) {
+        return this._layerVisibility ? this._layerVisibility[layerName] !== false : true;
+    }


I understand that this code is very helpful to build your example page (which is great by the way!). I think that this is something that should be managed by the Layer, not the source. What you are actually doing here is modifying source.layers in order that the parser filters the unwanted layers. You are updating the layer filtering at the source and re-instanciating the layer, not updating visibilities. There are discussions about off-screen GPU rendering for rasterized tiles. I think this is the way to implement true layer visibility switching. For now, I suggest that you could keep all this code by moving it to your example page. What do you think ? @Desplandis I can't find any issue related to off-screen ColorLayer rendering, is there one existing already ? I think this PR gives another reason to work on this great idea.

airnez · 2026-01-16T16:20:52Z

packages/Main/src/Source/PMTilesVectorSource.js

+
+                    // If no layers were provided, try to auto-detect from metadata
+                    if (Object.keys(this.layers).length === 0 && metadata.vector_layers) {
+                        this._setupLayersFromMetadata(metadata.vector_layers);


It's great to be able to parse headers directly. Thank you for implementing that part. It makes iTowns easier to use :)

ymoisan · 2026-01-19T21:16:17Z

Thanx for the thorough review @airnez. I'll be honest up front : although I have coding experience, everything in this PR is "vibe coded". I could never think of implementing all this in JS, a language I don't work with normally. However, I knew what I wanted so I guided the implementation best I could.

So, I submitted your comments to AI :

<begin-AI>
I completely agree with all your comments. Here's what I plan to do and a few questions:

What I'll fix:

Redundant code (lines 76, 86, 221, 234): I'll remove isVectorSource, parser assignment, feature cache initialization, and getDataKey() - you're absolutely right that Source.js already handles all of this.
Layer visibility management (lines 140-184): I completely agree this belongs in the example, not the Source. As you suggested, I'll move all the layer visibility management code (setLayerVisibility, setAllLayersVisibility, getLayerVisibility, getLayerNames, and the _originalLayers/_layerVisibility storage) from the Source to the example file.
Error handling (lines 263, 274): I'll fix both - return FeatureCollection instead of null and use this.handlingError(err) for errors.
Example optimizations (lines 135, 145): I'll create the style object once as a constant and remove the clearCache() call.

Questions:

1. Layer visibility UI implementation (addressing your comment on lines 140-184): I'd like to keep the layer visibility UI in the example since it's helpful for demonstrating PMTiles capabilities. As you suggested, I'll move all that code from the Source to the example. However, I'm wondering about the best approach for implementing this in the example - should I:

Create multiple ColorLayer instances (one per source layer from the PMTiles) and toggle their visible property?
Or modify the source.layers configuration in the example and re-instantiate the layer (which is what I'm currently doing, but moved to the example)?

Since you mentioned that true layer visibility switching will come with off-screen GPU rendering, what would be an acceptable interim approach for the example?

2. Error handling strategy (addressing your comment on line 274): You suggested using this.handlingError(). Should I always throw on error, or would returning an empty FeatureCollection be acceptable in some cases (e.g., when a tile doesn't exist)? The contract seems to be that loadData should either throw or return a FeatureCollection - is throwing always preferred?

3. Metadata parsing: Should I keep the automatic layer setup from metadata (_setupLayersFromMetadata) for initial layer configuration, even though I'm removing the visibility toggling parts? It's still useful for auto-detecting available layers when none are provided.

4. PMTilesSource parent class (addressing your comment on line 25): Yes, exactly! The parent PMTilesSource is intended to be reusable for future raster and elevation PMTiles implementations.

5. Broken tiles issue (addressing your comment about https://r2-public.protomaps.com/protomaps-sample-datasets/tilezen.pmtiles): I suspect the broken tiles might be related to the error handling issues you identified. Currently, the code:

Returns null when tiles don't exist (line 263) or when errors occur (line 275)
Caches these null promises (line 279), which might cause issues downstream if the code expects a FeatureCollection
Swallows errors instead of propagating them, making debugging difficult

Looking at how VectorTilesSource handles this (line 183), it uses this.handlingError(err) which throws, rather than returning null. Also, getTile() returns undefined when a tile doesn't exist (PMTilesSource.js line 88), but the check uses !data which catches both null and undefined, then returns null which might be inconsistent.

I suspect the broken tiles might be caused by:

Silent failures: Errors during parsing or fetching are caught and return null, but downstream code might not handle null properly
Cached null values: Once a tile is cached as null, it will always return null even if it becomes available later
Missing tiles: When a tile doesn't exist, returning null instead of an empty FeatureCollection might break rendering expectations

With the error handling fixes I mentioned above (returning empty FeatureCollection instead of null and using this.handlingError() for actual errors), this should be resolved. However, I'm wondering:

Should missing tiles (when getTile() returns undefined) return an empty FeatureCollection or throw?
Are there any specific error patterns you've observed with the protomaps tilezen.pmtiles file that might help me debug this?

<end-AI>

If that suits you I can provide code fixes. Otherwise let's continue the discussion.

- Remove redundant code (isVectorSource, parser, getDataKey, onLayerAdded) - Move visibility methods to example level (Layer responsibility) - Use this.handlingError for error handling - Return empty FeatureCollection instead of null - Simplify example: constant style, remove unnecessary clearCache

ymoisan · 2026-01-26T16:35:09Z

I tested my example and it works fine.

Summary of Changes

PMTilesVectorSource.js (simplified from 284 to 181 lines)

Reviewer Concern	Change Made
`isVectorSource = true` redundant	Removed - `Source.js` sets this based on format
`parser = VectorTileParser.parse` redundant	Removed - `Source.js` sets parser from `supportedParsers`
`getDataKey()` duplicates `Source.js`	Removed - uses inherited method
`onLayerAdded()` redundant	Removed - `Source.js` handles cache setup
`clearCache()` too low-level for Source	Removed - View/Layer responsibility
Visibility methods belong at Layer level	Moved to example page
Return `null` unsafe	Now returns `new FeatureCollection(out)`
Use `this.handlingError`	Now uses proper error handling pattern

airnez · 2026-02-23T11:11:13Z

@ymoisan Sorry for taking this long to answer. We talked about it a few weeks ago with other contributors, and I forgot to keep you in touch.
You're the first (I think) contributor that submitted an entirely "vibe-coded" PR to this project. This is ok (Progress doesn't stand still, they say). It is however important to us that it is mentionned in the PR message. We also think that contributors should be able to understand and apply PR review comments by themselves. We'll be updating our contributing guidelines soon.

For this one, I'll review your code and provide the code modifications if needed. Your contribution seems safe to merge because it's a new independant feature. I already spend time on it, let's push it. I'll then hand the final review to another maintainer. I'll keep you in touch :)

Thank you for contributing. You are welcome to contribute again, at the sole condition that you are able to understand provided code and apply review suggestions by yourself.

ymoisan added 2 commits December 10, 2025 09:52

New MVT PMTiles source with example

7afe87e

Source implementations

65648a3

airnez assigned ymoisan Jan 9, 2026

airnez self-requested a review January 9, 2026 09:21

airnez requested changes Jan 16, 2026

View reviewed changes

ymoisan changed the title ~~Add PMTiles MVT support~~ Add PMTiles basemaps to iTowns Feb 10, 2026


		const opacityToUse = userOpacity;

		pmtilesSource.clearCache();

Conversation

ymoisan commented Dec 19, 2025

Uh oh!

ymoisan commented Jan 6, 2026

Uh oh!

airnez commented Jan 9, 2026

Uh oh!

airnez left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ymoisan commented Jan 19, 2026

What I'll fix:

Questions:

Uh oh!

ymoisan commented Jan 26, 2026

Summary of Changes

PMTilesVectorSource.js (simplified from 284 to 181 lines)

Uh oh!

airnez commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

airnez commented Feb 23, 2026 •

edited

Loading