Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.95%. Comparing base (86ed382) to head (b869ce9).
Report is 194 commits behind head on main.

@@            Coverage Diff             @@
##             main    #4864      +/-   ##
==========================================
- Coverage   87.66%   86.95%   -0.71%     
==========================================
  Files         265      265              
  Lines       37937    37937              
  Branches     2407     2485      +78     
==========================================
- Hits        33257    32988     -269     
- Misses       3608     3828     +220     
- Partials     1072     1121      +49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

I think we should not merge a failing render test though, we should update the image or fix the root cause of this test as oart of this PR.

@NathanMOlson any updates on this PR? Can we move it forward somehow?

This PR can be moved forward by fixing #4859. I don't know how to fix #4859.

Also, a better solution than the one in this PR would be to update the pixelmatch library to include the changes here: mapbox/pixelmatch#142. These changes have been approved by the maintainer, but have not been merged because they add processing time. I do not see a way to fix the pixelmatch library without adding processing time. In my opinion, MapLibre would be better served with a version of pixelmatch that detects transparency differences, even if it is slightly slower.

In the past, a similar problem was solved by adding a border around the tiles to stitch different zoom level for tiles when terrain is on.
Is it possible that the border needs to be "bigger" to solve this issue (the tile is very far away).
I'm not sure it's the right fix, but wondering if it works solve the issue...

mapbox/pixelmatch#142 has been merged in the pixelmatch library.

I believe I addressed the change here:

• Bump pixelmatch from 6.0.0 to 7.1.0 #5541

Is there anything else needed as part of this PR?

@HarelM #5541 updated several render test so that the expected result includes visible defects.

high-pitch/pitch95 has 3 white pixels near the horizon in the upper left.
terrain/fog has a transparent section near the horizon in the upper right
terrain/symbol-add-terrain the tile "skirt" is visible

It feels wrong to test against defective results, would it be better to remove/disable these tests? I think the fact that prior to #5541, the expected results didn't have the defects, suggests that these are regressions.

#5541 fixes the problem that I was pointing out with this PR, which is that pixelmatch could pass render tests that should fail. However, it does not fix the defective terrain/fog rendering.

I'm fine with this being closed, or left open as a demonstration of the rendering defect in terrain/fog.

The render tests simply define the current behavior, if there's a fix that changes the results of the render test, we'll update the image, that's fine.
It's even better to see that if a fix was done the relevant render test works change for the best.

As for skipped tests, there were a bunch of skipped tests when I started maintaining this library, and I removed all of them as keeping skipped tests doesn't really help anyone.

If you want to indicate there's a behavior that needs fixing, an issue with a simple reproduction (using the style.json for the render test even) would be a much better approach.