WGSL/WebGPU GLSL ShaderGenerator

[scotbrew]

Adding WgslShaderGenerator and its support classes to generate WGSL-compliant GLSL code. This is targeted to allow USD to render MaterialX shaders with the WebGPU implementation.

Main changes:

1. LightData.type renamed to LightData.light_type -- "type" is a reserved word in WGSL. The change only affects the WGSL flavor of GLSL.
2. Changing the function signature and calls for all functions that use (sampler2D) to (texture2D, sampler), as WGSL GLSL is more restrictive and does not support sampler2D.

Pull Request Sections:

1. WgslShaderGenerator and its support classes
2. WGSL-compliant GLSL shaders (filenames: *_wgsl.glsl) (using #define HW_SEPARATE_SAMPLERS in glsl shader code)
3. Small modifications to base classes or refactoring a few .glsl files. and adding 2 baseclass virtual functions.

Issues Resolved:
This PR addresses the following logged issues.

• WebGPU/WGSL: Compiler Error, LightData struct member variable "type" is a reserved word. #2377
• WebGPU/WGSL: Option for GLSL Vulkan shader functions to pass separate texture2D and sampler parameters, instead of only sampler2D. #2379

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: scotbrew / name: Scot Brew (770fcaa, 7592972, 1eaa79d, 17666a4, 58208f2, 2fae1f6, 75cddf1, c32a2e4, 3216a40, 38867fd, fabd07f, 03eacd4, 131d322, 5622026, e16dc86, 8249985, 06ee2ed, c1a6c29, 84c011b, 44e5a48, aac3bfc, 2885f70, cc5f77d, a99de01, c50abbb)
• ✅ login: jstone-lucasfilm / name: Jonathan Stone (d94732f, a4a26c4, 9971db0, e7bbffb, 9a7c5c8)

[scotbrew]

No testing done with Js currently. Likely something would need to change around the LightData struct so it can use LightData.light_type instead of LightData.type to have Js work with WebGPU/WGSL.

[ashwinbhat]

Hi @scotbrew greatly appreciate you getting this work started 👍🏽 I briefly looked at the changes and think you are addressing all the key areas wrt the sampler split. However, I see that we will have to duplicate major blocks of shader code for minor differences in function call.
While having separate _wgsl.glsl and .glsl allows us to tweak the wgsl version later, I'm wondering for the initial release if we could consolidate mx_environment_* and hex_tile by using a compile time macro. So something like this:
#define VULKAN 100
#if VULKAN > 100
mx_latlong_map_lookup(Lw, $envMatrix, lod, $envRadiance_texture, $envRadiance_sampler)
#else
mx_latlong_map_lookup(Lw, $envMatrix, lod, $envRadiance)
#endif

Could we introduce a hwSeparateSamplers option in https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/source/MaterialXGenShader/GenOptions.h that allows this branching and emits the required #define. This will also reduce the need to create a separate WgslResourceBindingContext since Vulkan ResourceBindingContext could simply choose to sperate or combine based on hwSeparateSamplers option setting.

Based on suggestion from Niklas slack I think we want to use genwgsl instead of genglsl_wgsl so this will allow underlying implementation to change and remove the glsl notion in future.

[scotbrew]

@ashwinbhat , the #define is a great idea for a way to minimize the branching of the code.

I was not seeing a way for the WgslShaderGenerator to specify the GenOptions from the context in the constructor. We can leave adding a GenOption as a phase2 if wanting to pull it back to Vulkan.

void WgslShaderGenerator::emitDirectives(GenContext& context, ShaderStage& stage) const
{
    VkShaderGenerator::emitDirectives(context, stage);
    if (genContext.getOptions(). hwSeparateSamplers) {
        emitLine("#define HW_SEPARATE_SAMPLERS");
    }
}

[kwokcb]

1. i was hoping we could virtualize or modularize the lookups more but the raw code is pretty ugly. Even though there is duplication I'm not really a big fan of compiler directives in code, but I guess that's good enough for now. I was hoping that the image functions at least had emitted headers to branch on but that does not seem to be the case.

2. I'd prefer virtualization instead of introducing tests for a specific generator.

3. It should be pretty straight-forward to add in genwgsl test and that should give you pretty good base-line unit test coverate. If there is way to validate the code, then we can look at adding something to CI later on via the generateshader.py script. If you something that users can run to perform any validation would be good to document this.

[scotbrew]

I've combed over the changes to streamline a bit further. PR should be ready for final review.

[kwokcb]

This looks great with some minor items:

1. It would be good to expose the environment texture / sampler uniform names as a courtesy to implementors so they know what uniforms to use for binding.

I can't think of any other way to make the source code cleaner than what's done here with the preprocessor directive.

2. I think the sampler gradient call syntax is incorrect ?

Thanks.

[scotbrew]

Reviewed all comments and suggestions.

[JGamache-autodesk]

Apart from missing WGSL forbidden keywords in the Syntax, everything else looks fine to me.

[jstone-lucasfilm]

@scotbrew Oh, I just noticed that our render validation tests are returning all-black renders with this PR!

I've confirmed the same results locally on my Windows machine, and here are the current renders from the Viewer and Graph Editor with this PR:

Viewer:

Graph Editor:

[jstone-lucasfilm]

Looking back through recent GitHub Actions runs, here's the first one I see where the render validation tests are returning black renders, in case that helps to track down the cause:

https://github.com/AcademySoftwareFoundation/MaterialX/actions/runs/15331741009

[scotbrew]

That is definitely one to catch and fix before fully integrating.

Updates:

1. I'll take a look to repro it on my end and compare shader code to identify the fix.
2. Verified locally that the latest MaterialXView with this PR has black renders. Tracking down the introduced issue.
3. The issue has been isolated to a refactoring change for the PR in the commit "WGSL: Refactor shader code and CompountNode shader gen" (SHA: 131d322 ).
4. Fixed in the latest commit. The refactoring of ShaderGenerator::emitFunctionDefinitionParameter() needed to treat input and output ShaderPorts slightly differently.

[scotbrew]

The identified render issue regression has been addressed. Here are the latest images from the Linux test runners.

[scotbrew]

For code spot-checking, a comparison was done between the standard_surface_look_brass_tiled shader code generated by the latest PR2407 code and latest main branch. Everything looks as expected.

• VS: Identical
• PS: Diff adding only code scoped by #ifdef HW_SEPARATE_SAMPLERS. (see attached *-DIFF.txt)
• MDL: Identical

standard_surface_look_brass_tiled_ps-DIFF.txt

[jstone-lucasfilm]

Thanks so much for this contribution, @scotbrew, and I believe this looks ready to go!

Further in the future, I'd be interested in trying additional strategies to reduce the interface duplication in the hardware implementations of our image nodes, but the benefits of this initial implementation greatly outweigh those minor issues.