Switching to OpenMapTiles and Fixing Small Bugs

[ionmeo]

This description follows a [problem] => [commit link] format where each problem's description followed by the commit that resolves it.

Runtime Error

After cloning the repo, when you run npm install followed by node main.js, you will encounter:

TypeError: Cannot read properties of undefined (reading 'slice')`

To fix it, we need to always return the buffer in the _getHTTP method of TileSource.js.

commit 1736dac: fixed undefined slice error at runtime

Remote Tile Source Issues

After fixing the initial runtime error, running node main.js reveals rendering bugs at certain zoom levels.

These issues stem from problems with files hosted on mapscii.me rather than the codebase itself. You can verify this by using a local .mbtiles file. As osm2vectortiles was archived a while ago, you can only download such .mbtiles from Wayback Machine.

• Planet from z0 to z5 (20MB)

• Planet from z0 to z8 (411MB)

However, when running

node main.js --tile_source <path to world_z0-z5.mbtiles>

You will encounter Error: no TileSource defined. To fix the issue, we need to make the init method in TileSource.js and _initTileSource method in Mapscii.js asynchronous.

commit ea69053 : fixed no tilesource defined error when using local mbtiles file

Zoom Level Crashes

When using the .mbtiles files linked above, if you keep on zooming, the program crashes with Tile does not exist error as the file only supports zooming upto a certain point.

commit 5a1d9b0 : prevent zooming beyond max zoom level in local mbtiles file

Migrating to OpenMapTiles

The main three changes are:

• Layer Name Updates

  • Updated layer names _generateDrawOrder function in Renderer.js
  • Updated layers array in config.js.

• Change tile source

  • Updated source in config.js and TileSource.spec.js
  • Currently uses OpenFreeMap because using OSM US gives Unimplemented type: 4 error.
  • The source link is https://tiles.openfreemap.org/planet/map/ but the map part is a wildcard.

• Update styles in dark.json

  • Conversion details are in NOTES.md. Although it is quite long due to including both new and old json blocks, some repetitive patterns make it a quicker read than the file size suggests.

Commit ea03d52 : migrate to OpenMapTiles

Filter Interpreting Issue

At this point, when you zoom in, you will see color of the road layer has changed from yellow to pink.

To find the reason, we need to go back to commit 5a1d9b0 i.e. the codebase before migrating to OpenMapTiles. Now, take a look at the first two instances of "source-layer": "road" in the old dark.json.

{
  "type": "line",
  "id": "tunnel_path_pedestrian",
  "paint": {
    "line-color": "@tunnel_path_pedestrian"
  },
  "source-layer": "road",
  "filter": [
    "all",
    ["==", "$type", "LineString"],
    [
      "all",
      ["==", "structure", "tunnel"],
      ["in", "class", "path", "pedestrian"]
    ]
  ]
},
{
  "type": "line",
  "id": "tunnel_motorway_link",
  "paint": {
    "line-color": "@tunnel_motorway_link"
  },
  "source-layer": "road",
  "filter": [
    "all",
    ["==", "structure", "tunnel"],
    ["==", "class", "motorway_link"]
  ]
}

If you change the value of @tunnel_path_pedestrian, you won't notice a change in the map. However, if you change value of @tunnel_motorway_link, you will see a change in the color of all roads. This is problematic because the filter should color a road only if it's a tunnel and a motorway_link but it seems like it is coloring all the roads.

@tunnel_path_pedestrian has been renamed to @tunnel_path because in OpenMapTiles, pedestrian subclass is included in class path.

However, in the new version, when you modify the value of @tunnel_path, it is changing color of everything in transportation layer. This is most likely because the filter does not have double nesting anymore, implying an issue with interpretation of double nesting inside a filter. Fortunately, we do not have to fix it right now because we can write the same filter condition without using double nesting due to filter conditions being essentially just AND logic,

{
  "type": "line",
  "id": "tunnel_path",
  "paint": {
    "line-color": "@tunnel_path"
  },
  "source-layer": "transportation",
  "filter": [
    "all",
    ["==", "$type", "LineString"],
    ["==", "brunnel", "tunnel"],
    ["==", "class", "path"]
  ]
},
{
  "type": "line",
  "id": "tunnel_motorway_ramp",
  "paint": {
    "line-color": "@tunnel_motorway_ramp"
  },
  "source-layer": "transportation",
  "filter": [
    "all",
    ["==", "brunnel", "tunnel"],
    ["==", "class", "motorway"],
    ["==", "ramp", 1]
  ]
}

However, we do have to fix the issue of all transportation paths being colored with the first instance of layer transportation, regardless of the filter condition. This issue stems from handling of the 'all' keyword in Styler.js.

As the styles are now rendering properly, we can proceed to fix some issues with styling. As the map becomes cluttered with country names and boundaries at early zoom levels, I added some zoom limits via minzoom in dark.json. Additionally, I changed @admin_level_2 from #fff to #aac to reduce eye strain.

commit 1015ecd: fixed handling of filter keyword 'all' and updated styles

Polygon Rendering

There are still some rendering issues like blacked out areas where there is supposed to be ocean, and a huge water block appearing across Europe. To fix these issues, you have to modify the polygon rendering in Canvas.js. In the new version, we first separate between outer and inner rings and then use point-in-polygon to figure out which inner ring belongs to which outer ring.

commit d85535e : fixed water appearing over land and areas with missing water

Water Block Blackouts

Now, another issue you will notice is when zoomed in a bit, certain water blocks black out. You need to increase the range in _getTileFeatures function in Renderer.js to fix the issue.

commit ecda55a : fixed empty blocks at some zoom levels

Adapt bright.json

Finally, I just converted bright.json to be compatible with OpenMapTiles.

commit 588ec3c : adapted bright.json for OpenMapTiles

[quincylvania]

Yeah OpenFreeMap is probably the way to go for now. But the endpoint can be changed as needed since OpenMapTiles is an open standard.

[jaller94]

Looks good. It's updating the map and fixes a handful of issues correctly.

I'm not sure about the increased search area for elements. How does this make sure that we'll find all relevant points? I think it will still miss many. It might affect performance, but while testing it the performance seemed unchanged.
ecda55a

Self-hosting the tiles from Open is possible and legal. The single-person project offers new world files every week.

The adaptions to the styles are a bit too much for me to review by reading them. Opening the application shows that the style looks great and mostly unchanged to the current version on main.

A comment about something potentially breaking support on Windows seems unrelated. This PR does not touch anything related to the dependencies, launch or terminal output.

[ionmeo]

I looked into commit ecda55a and the issue was _addBoundaries in Tile.js used data.points[0] (only the first ring) to compute the fill bounding box instead of all rings. So, I changed data.points[0] to data.points.flat() which flattens all rings into a single array so the bounding box loop covers every point, and removed the searchPadding fix.

On Windows, running npm install -g . followed by mapscii fails with /bin/sh.exe is not recognized. So currently the only way to run on Windows is by cloning the repo and running node main.js. To fix this issue, we need to create bin/mapscii.js:

#!/usr/bin/env node
'use strict';
require('../main.js');

and change "./bin/mapscii.sh" to "./bin/mapscii.js" in package.json. As mapscii.sh is not used anymore, I removed it.

[jaller94]

@ionmeo Thank a lot for looking into this.
Could you please add yourself to the AUTHORS file?

Based on the writing style of the PR description, I assume that you've used an LLM. While that's probably ok, could you please disclose whether you used an AI assistant and which one?

[ionmeo]

After writing the draft pull request, I used Claude to rephrase some of the sentences, so some parts might seem like AI-generated. I took help from Claude for some of the bug fixes, most notably commit d85535e is almost entirely done by Claude. However, almost all of the code and json styles related to OpenMapTiles migration were written by me. Additionally, all changes made by Claude were checked to remove unnecessary code changes and verify no new bugs were introduced.