EXT_mesh_primitive_edge_visibility

[pmconne]

The EXT_mesh_primitive_edge_visibility extension augments a triangle mesh primitive with sufficient information to enable engines to produce non-photorealistic visualizations of 3D objects with visible edges. The edge visibility is encoded in a highly compact form to avoid excessively bloating the glTF asset. Adapted for glTF from iTwin/itwinjs-core#5581. We intend to add support for the extension to the CesiumJS engine.

[pjcozzi]

@pmconne so excited for this!

• @lilleyse could you please take a quick look from a 3D Tiles and CesiumJS perspective?
• @kring could you take a quick look from a Cesium Native and game engine perspective?

[pjcozzi]

Is the BENTLEY_ prefix still TBA based on discussion in #2478?

[pmconne]

Yes.

[pjcozzi]

It would not need to be near-term, but is there potential that this extension could replace CESIUM_primitive_outline in the future?

[pmconne]

Yes, the hard edge visibility encoded by that extension is a subset of that encoded by this extension.

[pjcozzi]

@lilleyse if you agree, do you want to submit an issue to the 3d-tiles repo for future consideration to replace CESIUM_primitive_outline with this extension over time?

[pjcozzi]

It's an edge case (no pun intended), but should the spec define the value of bits when there are extra/unused bits at the end of the bitstream?

[pmconne]

Added requirement that unused bits are zero-initialized.

[pjcozzi]

I don't know if I have a strong opinion but others may ask about the oct16 encoding and, if instead, this should be uncompressed and then able to use meshopt, etc. Would that fit into the ecosystem better and have similar characteristics?

[pmconne]

Yeah, I went back and forth on it and decided to leave as-is initially pending feedback. I note that while oct-encoded normals get some use in 3D Tiles 1.0, the closest thing in glTF is KHR_mesh_quantization (min 1 byte per component, plus padding for byte stride).

[pjcozzi]

Gotcha, let's see what others think. Also note the octahedral filter in EXT_meshopt_compression.

[pmconne]

The revised extension leaves it up to engines to compute the normals, obviating the need to specify their encoding.

[pjcozzi]

Same comment as above.

[pjcozzi]

Excellent to see these references!

[kring]

Thanks for opening this @pmconne! I have a few thoughts below.

Overall it looks good! I can't see any particular reason it couldn't be implemented in Unreal Engine or Unity.

[kring]

This file needs to be renamed to match the new extension name.

[javagl]

Bump @pmconne (triggered by the request for comments below).

[kring]

The rendering technique CesiumJS uses for CESIUM_primitive_outline has the opposite requirement. The edge needs to be known when rendering both triangles. So if only one is flagged, then the rendering engine will have to search for another triangle sharing that edge, which is a pretty expensive thing to do.

Maybe we don't need that technique at all anymore? I don't know. The nice thing about it is it doesn't require any separate draw calls, does no depth manipulation, and has no possibility of depth testing errors (because the lines are drawn on the edges of the triangles while rendering the triangles normally). All of which is quite nice for Cesium OSM Buildings. The downside is it's pretty inflexible in how the lines are styled.

If we do want that rendering technique to be a valid option, one idea is to use another "edge visibility" value to indicate "the other side of a shared hard edge". But going from two bits per edge to three would be expensive.

[pmconne]

That technique won't work for silhouettes, nor can it provide consistent edge widths (an important related requirement for AEC) since it can only draw inside the triangles. But I don't want to dictate specific implementations. I have changed the meaning of visibility value 3 to indicate a repeated hard edge.

[kring]

I think that it would be wise to specify the appearance of the unstyled edges. Clients with a good reason to can of course still override this and do something different. But by default, models should look the same in different clients supporting this extension.

[pmconne]

The extension now specifies the default material and permits alternate material(s) to be defined.

[kring]

Ok, so we don't want to use just a normal glTF LINES primitive to render these styled edges, because a normal glTF LINES primitive doesn't have any implication that the lines are coincident with triangles. Nor does it require that the lines and triangles avoid depth fighting. So in a client that doesn't support this extension, it's probably better to avoid rendering the lines entirely, rather than rendering them poorly. I made the same argument in CESIUM_primitive_outline and it makes sense here, too.

However, it seems very odd to me that styled lines are specified redundantly in two places: in the visibility accessor as value 3, and also in the primitives property. Why not use visibility value 0 for these non-automatic lines?

Or, better yet, why not allow the "compact" edge definition to be styled somehow? It's nice that you can use a compact representation for default-styled edges, but as soon as you want to customize their appearance, you're stuck with the full-size representation. One idea, without having thought about it too much, is to let the extension associate ranges of edges with materials. So styled edges would simply use visibility value 1 and then we'd have something like:

"materials": [
  {
    "material": 1,
    "ranges": [0, 10, 111, 187]
  },
  {
    "material": 3,
    "ranges": [11, 110]
  }
]

In the ranges properties, each pair of numbers is a start and end edge index, numbered according to their order in the visibility accessor.

Ranges could also be binary-encoded in an accessor instead of JSON, if that seems worthwhile.

This isn't ideal because there'll be a tension between ideal triangle ordering and grouping same-styled lines together, but I think it'll still be a big compactness win over a normal glTF LINES primitive even if no triangle reordering is done at all.

[kring]

Side benefit: if we don't need visibility value 3 for styled edges, then perhaps it could be used to indicate the other side of a shared hard edge, as I mentioned above.

[pmconne]

I have replaced primitives with lineStrings, and those edges are no longer encoded in visibility. Please review the revised spec.

[kring]

I looked briefly at the FeatureSymbology.ts code. Have you found this to do a good job of avoiding depth fighting between lines and triangles at all scales, while also avoiding artifacts where depth manipulation causes the lines to "bleed through"? If so (and maybe even if not), I think it would be great to include some implementation guidance in the spec, even if it's non-normative. I've found this to be challenging in the past, and that's why I eventually went with a very different technique for the outlines of Cesium OSM Buildings.

[pmconne]

Yes, we invested a lot of effort into ensuring the edges consistently display in front of their surfaces (and with consistent pixel width). I think we might want to rough out an implementation in CesiumJS before writing up implementation guidance here.

[kring]

It strikes me that this is a mesh primitive inside an extension on a mesh primitive. Should there at least be some requirements imposed on this, such as that the POSITION attribute of all of these inner primitive definitions must match the outer one?

[javagl]

I haven't looked at all this on a technical level, but looking at this point in isolation, I think that (it could pose some challenges for code generators, due to the "recursion" through an extension, and) it could be worthwhile to point out that this mesh primitive may not itself contain another instance of the extension 😬

[pmconne]

I have removed this quirk.

[pmconne]

@kring @javagl @pjcozzi thank you for your feedback. I've revised the spec accordingly.

[pjcozzi]

@kring @javagl @lexaknyazev or others - do you have any additional feedback for @pmconne on this extension?