WebGLRenderer: Add support for multiple uv sets and transforms.

[mrdoob]

Fixes #12608
Fixes #9457

Description

Pushing what I have for now. Still trying to figure out how to make this work.

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

Filesize	Gzipped	Diff from dev
627.4 kB	156 kB	+0 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

Filesize	Gzipped	Diff from dev
419 kB	102.1 kB	+0 B

[hybridherbst]

I love that there's progress here!

Related:

• GLTFLoader does not support texCoord fields #12608
• Support per-map per-Material UV channels, UV offset/repeat and texel scale/offset #8278
• Support for glTF models with two UV layers google/model-viewer#3515

[mrdoob]

Some progress:

https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/MultiUVTest

Before	After

[LeviPesin]

Maybe we can do something like

const mapKeys = [ 'map', 'emissiveMap', 'lightMap', otherMaps ];
for ( const key of mapKeys ) {
    parameters[ key ] = !! material[ key ];
    parameters[ key + 'Uv' ] = !! material[ key ] && getUVSetVar( material[ key ].uvSet );
}

and similar in other places? Just to simplify the code.

[mrdoob]

Yeah, I'll leave that for another PR though.

[LeviPesin]

I can do such PR (cleaning parameters and array generation in WebGLPrograms and defines generation in WebGLProgram).

[mrdoob]

Trying to add multiple texture transforms while I'm at it.

https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/TextureTransformMultiTest

Before	After

[mrdoob]

Managed to get webgl_loader_gltf_sheen to render correctly (finally!):

Before	After

[mrdoob]

I'm going to merge this and release today. If we find issues I'm happy to do patch releases.

[Mugen87]

I've noted in the migration guide that the uv2_* shader chunks have been removed. I've had a support case where custom shader code relied to it.

[mrdoob]

I've tested some examples with a Pixel 4a and couldn't see FPS differences, too. Although one probably needs a special test case with many textures to see a potential difference.

Yeah, I tried with models with a bunch of maps too and didn't see a difference.

I believe this has been one of the premature optimisations that have damaged the project the most... 😬

Feels pretty good to be outside of that hole now though!

[mrdoob]

@donmccurdy @elalish

Right now WebGLRenderer and GLTFLoader only supports 2 uv sets.

https://github.com/mrdoob/three.js/blob/dev/src/renderers/webgl/WebGLPrograms.js#L38-L44
https://github.com/mrdoob/three.js/blob/dev/src/renderers/webgl/WebGLPrograms.js#L154
https://github.com/mrdoob/three.js/blob/dev/src/renderers/webgl/WebGLProgram.js#L530
https://github.com/mrdoob/three.js/blob/dev/examples/jsm/loaders/GLTFLoader.js#L2033-L2034

Adding support 3 or 4 is pretty straight forward though.

Have you ever seen a gltf file with more than 2 uv sets?

[JonathanIcon]

Hello @mrdoob,
please find attached an example of the kind of GLTF files we would like to be able to visualize; we are stuck implementing THREEJS because of this limitation.
sphere.zip

[donmccurdy]

@mrdoob I've certainly seen glTF files with 3+ UV sets. Usually one or more of the UV sets are unused, I don't know enough about the art workflow to think of cases where 3+ UV sets would be needed. To my understanding Blender, Godot, Unity, and Unreal Engine each support up to 8 UV sets, but various conditions (Shader Graph in Unity, Skinning in Unreal, perhaps more...) will knock that limit down to 4 or fewer. I think supporting 4 UVs would be a nice bit of flexibility, if we aren't hurting performance.

@JonathanIcon do you mind explaining what this file does? Is the combination of MSFT_lod and KHR_materials_variants (both contained in the file) important to using additional UV sets here?

[JonathanIcon]

@donmccurdy we can ignore the MSFT_LOD yes but we use the KHR_materials_variants having 1 variant per material and each material has a different UV set;

[pailhead]

I think good question here would be:

Can three.js exist without GLTF?

Or

Is there one, or ten users perhaps, that use three.js without GLTF?

If three can in any scenario exist or be used without GLTF perhaps it shouldn't be the only consideration when designing this feature. Ie. "GLTF has only two channels, let's design three so that it uses only two channels".

If i may suggest an alternative pattern:

const g = new BufferGeometry()
g.setAttirbute('uv_0', ...)
g.setAttirbute('uv_1, ...)
g.setAttirbute('uv_2', ...)
g.setAttirbute('uv_3', ...)

If we consider these uvs as just array of attributes that exist without GLTF, OBJ, dae, 3ds and whatnot, i think it may become obvious that 2 channels is rather arbitrary :)

[pailhead]

This is, is how this editor looks in 3ds max:

The 1:map is basically what three calls uv . In practice, this is just the same mesh copied over and over, just like three's attributes. There has to be a limit, usually 32 for stuff like this in max, but in theory a user can add a whole bunch of mapping channels. Maybe even looking up the same texture, but from different UVs.

Feel free to ping me, i've worked with 3d studio max and other sowftware for 20 years before i thought myself how to program.

[pailhead]

Hm, it's also worth noting that 3ds max treats all of this as UVW. So in theory i think at any moment you can designate something as UV kinda like:

g.setAttribute('uv', g.getAttribute('position'))

So in a sense (i don't know much about GLTF) it can be even more generic? I imagine that GLTF saves space and wouldn't write a bunch of zeros for UVW, but write two numbers for UV. Thus probably making something like this incompatible with three.

However:

const loader = new GLTFLoader()

loader.load(['position.gltf', 'uv0.gltf', 'uv1.gtlf', 'uv2.gltf', 'uv3.gltf']).then(results=>{
   const g = new BufferGeometry()
   g.setAttribute('position', results[0]...)
   g.setAttribute('uv_0', removeZ( results[1]...)
   ...
})

Should be pretty generic if applying the same pattern. Any mesh, as long as it's the same can be combined into a geometry?

[pailhead]

@mrdoob I've certainly seen glTF files with 3+ UV sets. Usually one or more of the UV sets are unused, I don't know enough about the art workflow to think of cases where

The art flow from an engineering point can be seen as very generic. It's a creative process, and there may be a way to use several UVs while reusing several textures to create an effect that no one has thought of before. We are limited by the number of attributes in GL land, but when considering that a technical artist can be involved, i wouldn't find it unreasonable for someone to have a need to use say 8 channels, while using no attributes for skinning or perhaps even normals.

I think the channel info tool from 3ds max is a good example. (it's several meshes of 960 triangles, combined into one)

[donmccurdy]

Many devices are limited to 16 vertex attributes. In theory we could support up to 15 UVs + 1 position attribute, but in practice it may reduce complexity to have some limit. We are not designing around glTF here — it has no limit either — just trying to compare notes on use cases, and how often we see them.

[pailhead]

Correct, so maybe even something like:

const g = new BufferGeometry()
g.addUVChannel( new BufferAttribute(...) )
g.addUVChannel( new BufferAttribute(...) )

This wires it to three's uv uv2 or whatever is the representation, whereas maybe it wouldnt make sense, but would be allowed to do

const g = new BufferGeometry()
const channel = new BufferAttribute(...)
g.addUVChannel(  channel )
g.setAttribute('horse', channel)

[mrdoob]

#25788