Shader compilation regression starting from [v4.6-dev6] up to [v4.6.1-rc1]

[rei-00]

Tested versions

Regression exists starting with [4.6-dev6]
Personally tested with versions below. But highly likely that any version after [4.6-dev6] has this.

Reproducible: [4.6.1-rc1], [4.6-stable], [4.6-rc1], [4.6-beta1], [4.6-dev6].
Not reproducible: [4.6-dev5], [4.6-dev3], [4.6-dev1], [4.5.2-rc1], [4.5.1-stable], [4.5-stable].

System information

Godot v4.6.dev6 - Linux Mint 22.2 (Zara) on X11 - X11 display driver, Multi-window, 1 monitor - Vulkan (Forward+) - dedicated AMD Radeon RX 6700 XT (RADV NAVI22) - AMD Ryzen 7 5800X 8-Core Processor (16 threads) - 31.23 GiB memory

Issue description

After migrating my project from [4.5.1-stable] to [4.6-stable] some functionality in my game began to exhibit stuttering in the form of FPS drops. These are some of the examples:

1. Showing VoxelGI during runtime (expected to lead to FPS drops, but it was previously not noticeable)
2. Using ResourceLoader.load_threaded_request() on scenes (also expected, but did not lead to noticeable stutters before)

While testing my project between the versions, it seems that there is about a 5x increase in process time relating to the above shader compilation tasks. A compilation that would increase process time to 5ms in [4.5.1-stable] ends up taking 25ms in [4.6-stable] with all variables controlled for.

I've attached an MRP to demonstrate this with VoxelGI. I've not been able to recreate this for ResourceLoader consistently outside of my game project as my code is a mess and there are too many variables to consider.

Steps to reproduce

1. Run the MRP attached in [4.6.dev6]
2. Observe and note the FPS and Process values as VoxelGI turns on
   (on my system: FPS: ~46 | Process: ~0.26)
3. Run the MRP in [4.6.dev5]
4. Re-bake VoxelGI
5. Observe the same values. Process time is expected to reduce by 5 times
   (on my system: FPS: ~60 | Process: ~0.06)

Minimal reproduction project (MRP)

MRP.zip

[clayjohn]

Seems like it might be caused by re-spirv which does increase the shader compilation time of cached shaders by a little bit (although 5x is much more than any of us observed during testing).

To clarify, are you loading a voxelGI node at runtime?

This sounds like a case where the ubershader doesn't have knowledge of the voxelGI node until runtime and thus has to recompile all shaders once the voxelGI node is loaded

[rei-00]

I'm not sure what qualifies as runtime exactly... My technical knowledge on this is pretty limited still.

What I have is the VoxelGI enabled and visible in the editor before starting. Then on _ready() I hide() the VoxelGI node and show() it later. This used to not cause visible stutters in my project.

One fix I found that does not cause stutters is if instead of using hide() in _ready() I put voxelgi.data.energy = 0. Then when I want to show it I just set the energy value to what I need. This causes no stutter at all.

I hope this is helpful.

[clayjohn]

That is very helpful. Thank you.

[blueskythlikesclouds]

This might be salvageable by dumping the result of respv::Shader to the shader container binary, but I don't think loading VoxelGI mid frame is something you want to do either way. Doing that enables the advanced shader group, which causes either shader compilations to kick in, or file lookups to happen in the disk. It may be stutter free on old versions on your PC, but might not on potentially weaker systems.

[clayjohn]

Doing that enables the advanced shader group, which causes either shader compilations to kick in, or file lookups to happen in the disk. It may be stutter free on old versions on your PC, but might not on potentially weaker systems.

They say the VoxelGI node is in the scene tree at load time, so the advanced shader group should be enabled at load time and shouldn't create a stutter when unhidden.

This seems like a case where profiling will reveal the culprit very quickly