Assert/segfault on simple vertex shader

[Please-just-dont]

So I don't know if the syntax I'm using is correct because I'm only new, but the point is that slangc is failing the compilation and returning an error value, but not writing anything to stdout or stderr:

[[vk::location(0)]] in int2 inPos;


struct PushConstants {
    float point_size;
    float padding;
    float2 resolution;
};



[shader("vertex")]
void main(uniform PushConstants push_constants, out float4 pos: SV_Position)
{



    float2 wnd_position = float2( 1, 1 );
    float2 vertex_position = inPos.xy;
    float2 screenspace_pos = inPos.xy;
    

    
    screenspace_pos = wnd_position + vertex_position;


    
    float2 NDC_pos = float2(screenspace_pos.x / 3840 * 2 - 1, screenspace_pos.y / 2160 * 2 - 1);

    pos = float4(NDC_pos, 1, 1);


}

And here's the weird part, the bug goes away by deleting different parts of the code, so it's not one particular part of the code that makes it happen. Try commenting out different parts and see.

I'm using the slangc compiler that comes with the Vulkan SDK 1.4.304.1

[expipiplus1]

This fails in more than one way:

build/Debug/bin/slangc a.slang -stage vertex -target metal -> segmentation fault

build/Debug/bin/slangc a.slang -stage vertex -target spirv -> assert in glsl legalization

[sricker-nvidia]

Started investigating. Able to repro the metal and spirv cases. Issue seems related to mixing global variable inputs and a uniform struct as a param to a shader entry point. I created a smaller repro case:
`in int2 inPos;

struct PushConstants {
float point_size;
float padding;
float2 resolution;
};

[shader("vertex")]
void main(uniform PushConstants push_constants, out float4 pos: SV_Position)
{
pos = float4(inPos.x, inPos.y, 0, 0);
}`

Using void main(PushConstants push_constants, out float4 pos: SV_Position) (removing the uniform qualifier) compiles ok, although I did not validate the emitted spirv code. Similarly, using void main(uniform float push_constant, out float4 pos: SV_Position) works ok.

The bug may disappear when commenting out different parts of the code due to compiler optimizations. I've found that I can also get rid of the bug by not actually using inPos, for example I can use, pos = float4(0,0 0, 0); in the simplified shader to make the bug go away.

Also noting that there appears to be a second issue in the metal case, where mixing out and SV_Position leads to an issue in legalizeSystemValue() in source/slang/slang-ir-legalize-varying-params.cpp where permittedType will not = varType and will cascade into trying to emit for a kIROp_BuiltinCast instruction, for which there is no recognized case.

[sricker-nvidia]

I've managed to do some more research and I've been finding that this seems to be part of a larger more general issue where slang does not currently support mixing GLSL-flavored global in/out variables with entry point parameters. That is, something like:
in int2 inPos;
in int2 inTexCoord;
in int2 inNorm;

[shader("vertex")]
void main(int2 test1, int2 test2, out float4 pos: SV_Position)
{
// Use all the inputs to prevent optimizing out
pos = float4(inPos.x + inTexCoord.x + inNorm.x, test1.x, test2.y, 0);
}

and compiling it with -stage vertex -target glsl will get an output like:

...
layout(location = 0) in ivec2 test1_0;
layout(location = 1) in ivec2 test2_0;
layout(location = 0) in ivec2 translatedGlobalParams_inPos_0;
layout(location = 1) in ivec2 translatedGlobalParams_inTexCoord_0;
layout(location = 2) in ivec2 translatedGlobalParams_inNorm_0;
void main()
{
gl_Position = vec4(float(translatedGlobalParams_inPos_0.x + translatedGlobalParams_inTexCoord_0.x + translatedGlobalParams_inNorm_0.x), float(test1_0.x), float(test2_0.y), 0.0);
return;
}

It compiles, but the locations of the global in vars overlap with the entry point variables. Changing one of the entry point parameters will cause the asserts/seg faults that have been observed due to assumptions in slang source that assume that in parameters are not mixed.

I've spoken with some others about the issue and the consensus seems to be that we should make it an error to mix the two styles and that mixing global in vars with entry point variables should not be supported.

I plan to move forward on the issue by issuing an error/warning when slang detects the mixing such that things can fail gracefully instead of seg faulting or asserting. I will try to add a note in documentation as well.

[jkwak-work]

The PR link for this issue is:

• Address issues with GLSL style global in/out vars (#6669) #6998