WebGPU/WGSL: Option for GLSL Vulkan shader functions to pass separate texture2D and sampler parameters, instead of only sampler2D.

[scotbrew]

Request:
To support WebGPU/WGSL (translated from GLSL), GLSL Vulkan code option for needs separate texture2D and sampler binding handles passed in as function parameters instead of only sampler2D.

Description:
WebGPU/WGSL is stricter and can only bind separate texture2D and sampler, contrary to GLSL Vulkan which has options to bind a combined sampler2D (texture2D+sampler) as well as bind separate texture2D and sampler. There also looks to be a limitation in both GLSL and WGSL that prevents one from constructing a sampler2D struct (with texture2D and sampler) when one binds the texture2D and sampler separately.

To allow MaterialX GLSL Vulkan shader code to work with USD WebGPU, the generated MaterialX GLSL Vulkan code needs to be able to pass separate texture2D and sampler variables.

Steps to Reproduce:
With shader code using separate binding of texture2D and sampler, try to compile with glslang.
./glslang --target-env vulkan1.3

Original, using sampler2D: (works with GLSL Vulkan, but not WebGPU/WGSL)

layout(binding = 13) uniform sampler2D domeLightFallback;

void mx_image_color3(sampler2D tex_sampler, int layer, vec3 defaultval, vec2 texcoord, int uaddressmode, int vaddressmode, int filtertype, int framerange, int frameoffset, int frameendaction, vec2 uv_scale, vec2 uv_offset, out vec3 result)
{
    ...
}

void NG_tiledimage_vector3(sampler2D file, vec3 default1, vec2 texcoord1, vec2 uvtiling, vec2 uvoffset, vec2 realworldimagesize, vec2 realworldtilesize, int filtertype, int framerange, int frameoffset, int frameendaction, out vec3 out1)
{
    ...
    mx_image_vector3(file, 0, default1, N_multtilesize_vector3_out, 2, 2, filtertype, framerange, frameoffset, frameendaction, vec2(1.000000, 1.000000), vec2(0.000000, 0.000000), N_img_vector3_out);
    ...
}

Proposed Option, using texture2D and sampler:

layout(binding = 0, set = 1) uniform texture2D textureBind_domeLightFallback;
layout(binding = 0, set = 2) uniform sampler samplerBind_domeLightFallback;

void mx_image_color3(texture2D textureBind_tex, sampler samplerBind_tex, int layer, vec3 defaultval, vec2 texcoord, int uaddressmode, int vaddressmode, int filtertype, int framerange, int frameoffset, int frameendaction, vec2 uv_scale, vec2 uv_offset, out vec3 result)
{
    vec2 uv = mx_transform_uv(texcoord, uv_scale, uv_offset);
    result = texture(sampler2D(textureBind_tex, samplerBind_tex), uv).rgb;
}

void NG_tiledimage_vector3(texture2D textureBind_file, sampler samplerBind_file, vec3 default1, vec2 texcoord1, vec2 uvtiling, vec2 uvoffset, vec2 realworldimagesize, vec2 realworldtilesize, int filtertype, int framerange, int frameoffset, int frameendaction, out vec3 out1)
{
    ...
    mx_image_vector3(textureBind_file, samplerBind_file, 0, default1, N_multtilesize_vector3_out, 2, 2, filtertype, framerange, frameoffset, frameendaction, vec2(1.000000, 1.000000), vec2(0.000000, 0.000000), N_img_vector3_out);
    ...
}

Workarounds (and their limitations):

• Workaround1: USD WebGPU tries to workaround by using a #define. This works for some of the functions that use sampler2D directly. However, the GLSL compiler does not allow sampler2D with this construct to be passed in as a function parameter. This does not work for the shader code output from MaterialX.
  #define HgiGetSampler_domeLightFallback() sampler2D(textureBind_domeLightFallback, samplerBind_domeLightFallback)
• Workaround2: Unroll mx_image* functions into function calls to remove passing in sampler2D as a function parameter. This works for some of the MaterialX shader code samples like standard_surface_plastic.mtlx. However, this does not work on calls like NG_tiledimage_vector3() that take a sampler2D function parameter and then pass it into mx_image_vector3(), which is used by more complex materials.
• Workaround3: Regex string replace sampler2D with texture2d textureBind_, sampler samplerBind_ for NG_tiledimage_vector3 and similar functions. This allows for standard_surface_brick_procedural.mtlx to now work.

[ashwinbhat]

@scotbrew since the primary location that needs this change is the Vulkan Shader Generator, is the main requirement here "VkShaderGenerator should generate Vulkan 1.3 Syntax" ?

[scotbrew]

Good question. We are looking for an enhancement. Vulkan 1.3 syntax supports binding sampler2D directly, so the existing shader code output by MaterialX that expects sampler2D direct binding is already valid.
layout(binding = 13) uniform sampler2D domeLightFallback;

The ask is for the MaterialX shaders to support the alternative Vulkan 1.3 method to separately bind texture2D and sampler handles, and pass those handles separately into functions. From what I can ascertain, WebGPU/WGSL derived from Vulkan code only supports separate binding of texture2D and samplers.

layout(binding = 0, set = 1) uniform texture2D textureBind_domeLightFallback;
layout(binding = 0, set = 2) uniform sampler samplerBind_domeLightFallback;

From the Vulkan spec, one can reference the section "Combining separate samplers and textures", as Vulkan 1.3 supports both combined and separate texture sampler methods:
https://github.com/KhronosGroup/GLSL/blob/main/extensions/khr/GL_KHR_vulkan_glsl.txt#L351

If one tries to construct sampler2D(my_tex, my_sampler) from texture2D and sampler bindings and then pass that in as a function parameter, the glslang compiler throws the error below. I could not find a way around that.
ERROR: unknown:2917: 'call argument' : sampler constructor must appear at point of use

[ashwinbhat]

I used python python\Scripts\generateshader.py --target vulkan resources\Materials\Examples\StandardSurface\standard_surface_brass_tiled.mtlx to generate the attached shader.
Tiled_Brass.vulkan.frag.txt

Notice in addition to u_envRadiance u_envIrradiance we also have
layout (binding=5) uniform sampler2D image_roughness_file; layout (binding=6) uniform sampler2D image_color_file;
Based on your example and this,
void mx_image_float(sampler2D tex_sampler,...
should be
void mx_image_float(texture2D tex, sampler s,...

Unfortunately, I'm unable get glslangValidator.exe to do any error reporting if I set --target-env vulkan1.3
It looks like GL_KHR_vulkan_glsl leaves it to the front end tools to handle this?

[scotbrew]

Yes, that Tiled_Brass_vulkan.frag is valid Vulkan code as Vulkan supports both binding options (sampler2D and texture2D+sampler).

For WebGPU (as seen through USD), the binding method for the uniforms has to be texture2D+sampler as that is the only method supported by WebGPU (that it translates from Vulkan code). We need a way for MaterialX generated (Vulkan) shader code functions to be able to work with the uniforms texture2D and sampler. As far as I am aware, MaterialX currently only supports the combined sampler2D binding method.

I'm glad to get on a call if useful to discuss insights and options more interactively. We can then revise and clarify the Issue Request.

[ashwinbhat]

For the proposed changes would be something like this:

layout (binding=5) uniform texture2D u_envRadiance_t;
layout (binding=6) uniform sampler u_envRadiance_s;
layout (binding=7) uniform texture2D u_envIrradiance_t;
layout (binding=8) uniform sampler u_envIrradiance_s;

layout (binding=9) uniform texture2D image_roughness_file_t;
layout (binding=10) uniform sampler image_roughness_file_s;
layout (binding=11) uniform texture2D image_color_file_t;
layout (binding=12) uniform sampler image_color_file_s;

The implementation of image node will accept a sampler and texture. This might be tricky do to since the image node has them combined

void mx_image_float(texture2D t, sampler s, int layer, float defaultval, vec2 texcoord, int uaddressmode, int vaddressmode, int filtertype, int framerange, int frameoffset, int frameendaction, vec2 uv_scale, vec2 uv_offset, out float result)
{
    vec2 uv = mx_transform_uv(texcoord, uv_scale, uv_offset);
    result = texture(sampler2D(t, s), uv).r;
}

Tiled_Brass.vulkan-webgpu.frag.txt

[scotbrew]

Here are some working samples from theMaterialX standard_surface_brick_procedural.mtlx running in an experimental USD HdSorm WebGPU:

1. standard_surface_brick_procedural.glsl.ORIG.frag: (original) Uses sampler2D in functions. Is valid Vulkan code. Does not work when translated to WGSL.
2. standard_surface_brick_procedural.glsl.MODIFIED.frag: (modified -- generated from regex string replacement as a prototype) Uses texture2D and sampler in functions. Is valid Vulkan code and WGSL code.
3. Prototype image of this MaterialX shader compiled against WGSL running in USD in the Chrome browser.

standard_surface_brick_procedural.glsl.ORIG.frag.txt

standard_surface_brick_procedural.glsl.MODIFIED.frag.txt

[scotbrew]

Issue being addressed by #2407 .