Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement shader pipelines #1587

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

VReaperV
Copy link
Contributor

Requires #1586

This further improves the shader building by using GL_ARB_separate_shader_objects when it is available. This extension allows linking different shader stages separately, thereby decreasing the total amount of linking required.

As an example, on map station15 with all the shader stuff except liquid mapping and static reflections enabled, it went from 433 permutations/~866 compiled shaders/433 linked programs to 433 permutations/287 compiled shaders/286 linked programs, and cut the total amount of time it took to build shaders almost in half.

I'd had to add some ugly workarounds to make it work with both shader pipelines and regular shader programs due to how our GL uniforms work. Because of that this is currently draft, as I'm thinking of moving all these uniforms to a UBO when shader pipelines are available.

Also make sure to display the full shader name rather than just the main shader.
Previously, the GLSL shader building was done in the following manner:
1. When the renderer is started, some of the shader source was initialised/processed. This included the main shader text, #insert's, and material system post-processing.
2. When `buildAll()` was called, all permutations with empty deform shaders were built. The deform shaders themselves are loaded either at the start (for the default, empty one), or when the map itself is being loaded. These permutations included adding macros to shaders in the form of `#ifndef #define #endif`, and attaching the default deform shader. When possible, shader program binaries were saved/loaded to/from disk.
3. UI shader and non-0 deform shaders were built on-demand.

There were multiple issues with that system:
1. Shader caches were unreliable. This is likely because the checksum was calculated *before* macros and such were added to the shader text.
2. Every shader permutation was always compiled and linked every time.
3. Shader programs with non-0 deformVertexes were never cached.

This commit changes the underlying system to use an `std::vector` of shader program descriptors, and another one for shader descriptors. The initialisation/building of shaders is now more generic for each type, and as a result each unique shader is only compiled once. The permutations are still linked each time (unless downloaded from cache), but this change lays the foundation to easily add shader pipelines, which would solve this problem.

Shader cache binaries are now identified by each shader compiled into the corresponding shader program, which allows caching non-0 deformVertexes too.

This also fixes the incorrect #line numbers emitted by #insert due to macros and such being added after #insert is processed.

Also adds more information into how many shader/shader programs were built and how much time it took (there's currently a small discrepancy, because the output is shown at the end of `BuildAll()`, but is gathered every time `BuildPermutation()` or `InitShader()` is called).
@VReaperV VReaperV added A-Renderer T-Feature-Request Proposed new feature labels Mar 10, 2025
@VReaperV
Copy link
Contributor Author

I'd had to add some ugly workarounds to make it work with both shader pipelines and regular shader programs due to how our GL uniforms work. Because of that this is currently draft, as I'm thinking of moving all these uniforms to a UBO when shader pipelines are available.

This would be much easier if we drop support for hardware straight out of Pleistocene.

@illwieckz
Copy link
Member

illwieckz commented Apr 3, 2025

It looks like Mesa supports GL_ARB_separate_shader_objects on every hardware. I have verified Intel, Nvidia, ATI/AMD, Arm (Mali) and Broadcom (VideoCore), this included some of the oldest of each brand I own. I even seen this extension listed with something as crazily old as an Nvidia RIVA TNT2 (nv05), something we cannot support as we dropped the GL1 renderer 10 years ago anyway, and Mesa already supported it on it 5 years ago.. This extension is probably just a matter of API implementation, and is very likely totally unrelated to the hardware. It looks like OpenGL made it mandatory only with GL 4.1, but it looks like that doesn't prevented Mesa to implement it on older hardware, even hardware from Pliocene 😅️.

@VReaperV
Copy link
Contributor Author

VReaperV commented Apr 3, 2025

It looks like Mesa supports GL_ARB_separate_shader_objects on every hardware. I have verified Intel, Nvidia, ATI/AMD, Arm (Mali) and Broadcom (VideoCore), this included some of the oldest of each brand I own. I even seen this extension listed with something as crazily old as an Nvidia RIVA TNT2 (nv05), something we cannot support as we dropped the GL1 renderer 10 years ago anyway, and Mesa already supported it on it 5 years ago.. This extension is probably just a matter of API implementation, and is very likely totally unrelated to the hardware. It looks like OpenGL made it mandatory only with GL 4.1, but it looks like that doesn't prevented Mesa to implement it on older hardware, even hardware from Pliocene 😅️.

It doesn't make sense for GL1 cards, since they don't support the programmable pipeline anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants