Skip to content

Fix depth/stencil state for is3D drawables in layer groups without stencil tiles#4378

Open
Kotelberg wants to merge 4 commits into
maplibre:mainfrom
Kotelberg:upstream-pr/is3d-layer-group-state
Open

Fix depth/stencil state for is3D drawables in layer groups without stencil tiles#4378
Kotelberg wants to merge 4 commits into
maplibre:mainfrom
Kotelberg:upstream-pr/is3d-layer-group-state

Conversation

@Kotelberg

Copy link
Copy Markdown
Contributor

Problem

Layer groups that contain is3D drawables normally derive their depth (Metal) and stencil (GL) render state from the group's stencil tiles. That derivation was gated behind a check for non-empty stencil tiles, so when a group had is3D drawables but no stencil tiles at all - for example, a fill-extrusion group whose visible tiles happen to produce no stencil clipping work - the group state was never computed, and the per-drawable depth/stencil state that depends on it was never applied.

The failure mode differs by backend:

  • Metal: with no depth state applied, the 3D drawables render without depth testing. They paint over whatever is already in the color buffer regardless of actual depth ordering, so extruded geometry (e.g. 3D buildings) appears on top of content it should be occluded by.
  • OpenGL: the group's stencil state is left stale from whatever the previous group configured. With no explicit stencil op for the group, the leftover state causes the stencil test to discard every fragment the group draws, so the layer disappears entirely.

Both are silent failures - no errors or warnings, the layer just renders incorrectly or not at all, and only when this specific combination (is3D drawables, empty stencil tiles) occurs.

Repro

Any style with a fill-extrusion layer whose tiles produce no stencil work in the same layer group reproduces this. It's easiest to hit with a 3D buildings layer on the Drawable renderer: as soon as the visible tile set for that group contains no tiles needing stencil clipping, the group's depth/stencil state is skipped and the symptoms above appear (no depth test on Metal, fully discarded fragments on GL).

There is no existing automated test harness that exercises this specific layer-group/tile combination (a layer group with is3D drawables and zero stencil tiles), so this PR does not add a new automated test for it. I'd welcome guidance from reviewers on where such a test would best fit - it likely needs a render/gfx test that can construct a layer group with is3D drawables and an empty stencil-tile set for the target renderer.

Fix

The root cause was that group-level depth/stencil state was computed inside the same conditional branch as the stencil-tiles-non-empty check, so the two were coupled when they shouldn't be: is3D-derived state should always be computed for a group that has is3D drawables, while the stencil-tile-derived (stencil3d) state should stay gated
on stencil tiles actually being present.

This is split into two commits, one per backend, since each backend's state-tracking code is independent:

  1. Metal (src/mbgl/mtl/tile_layer_group.cpp): move the is3D/depth-state detection out of the stencil-tiles-non-empty gate so the group's depth state is computed whenever the group has is3D drawables, independent of whether it has stencil tiles. The stencil3d computation remains gated on stencil tiles being present, since that state genuinely only applies when there's stencil work to do.
  2. OpenGL (src/mbgl/gl/layer_group_gl.cpp): the same restructuring for the GL backend - is3D detection moves outside the stencil-tiles gate so the group's stencil state gets explicitly (re-)applied for is3D drawables even when there are no stencil tiles, instead of leaving a stale stencil configuration in place.

A third commit adds changelog entries for both platforms (platform/ios/CHANGELOG.md, platform/android/CHANGELOG.md) describing the fix.

Testing

  • Ran the full mbgl-test-runner suite: 976 of 988 tests executed (12 filtered, 4 disabled), 974 passed. The only 2 failures (MapSnapshotter.setStyleURL, MapSnapshotter.annotation) are pre-existing and reproduce identically on a clean upstream/main checkout with no changes applied — this PR introduces no new failures and no regressions.
  • Verified the fix on-device in a production app in both affected configurations: iOS (Metal) and Android (GL), confirming the 3D layer now renders with correct depth ordering / visibility in the previously-broken layer-group state.

Kotelberg added 3 commits July 4, 2026 13:27
Drawable::draw skips depth-stencil setup for is3D drawables, expecting the
layer group to bind the 3D depth state. TileLayerGroup only detected 3D
features when stencil tiles were present, so is3D custom-drawable geometry
rendered with no depth test at all.
Same gap as the Metal fix: DrawableGL::draw skips stencil setup for is3D
drawables, expecting the layer group to handle it, but TileLayerGroupGL only
detected 3D features when stencil tiles were present. is3D custom-drawable
geometry therefore inherited the previous layer's tile-clip stencil state and
all its fragments were discarded (cube invisible on Android GL).
@github-actions github-actions Bot added android iOS core Changes that affect the C++ core of MapLibre Native OpenGL Issues related to the OpenGL renderer backend labels Jul 6, 2026
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Bloaty Results 🐋

Compared to main

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +608  +0.0%    +524    TOTAL

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-4378-compared-to-main.txt

Compared to d387090 (legacy)

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +51% +59.1Mi  +467% +27.9Mi    TOTAL

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-4378-compared-to-legacy.txt

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results ⚡

Benchmark                                                     Time             CPU      Time Old      Time New       CPU Old       CPU New
------------------------------------------------------------------------------------------------------------------------------------------
OVERALL_GEOMEAN                                            +0.0019         +0.0022             0             0             0             0

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/benchmark-results/pr-4378-compared-to-main.txt

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Bloaty Results (iOS) 🐋

Compared to main

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +112  [ = ]       0    TOTAL

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results-ios/pr-4378-compared-to-main.txt


### 🐞 Bug fixes

- Fix missing depth/stencil render-group state for 3D drawables in layer groups without stencil tiles (previously: Metal rendered without depth, OpenGL discarded all fragments).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add it under a new ## main heading and link the PR.

Comment thread platform/ios/CHANGELOG.md
## 6.27.0

- Implement ambient cache for PMTiles sources ([#4290](https://github.com/maplibre/maplibre-native/pull/4290)).
- Fix missing depth/stencil render-group state for 3D drawables in layer groups without stencil tiles (previously: Metal rendered without depth, OpenGL discarded all fragments).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add it under a new ## main heading and link the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

android core Changes that affect the C++ core of MapLibre Native iOS OpenGL Issues related to the OpenGL renderer backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants