Running in Linux CI

[JustASquid]

I have run into a few issues trying to get the .NET tests running in Linux CI (Github Actions).

I see there is an action https://github.com/MikeSchulze/gdUnit4-action

However (please do correct me if I'm wrong here), this feels more suited to gdscript tests. Additionally, since I'm already installing Godot in order to export the project, I don't want to use this action as I would then download/install godot twice.

Ideally, I would run in CI using dotnet test as normal. However, this presents a few problems.

Headless vs xvfb

I have seen that there are issues in running Godot in headless mode due to input handing: godotengine/godot#73557
The gdUnit4-action works around this by running through xvfb, i.e. rendering to a virtual framebuffer.
This has a significant issue in my CI pipeline due to the fact that it is not running on a GPU instance (and to do so would be very expensive) as such, all the tests are VERY (i.e. unusably) slow as it tries to actually render everything.

As such I would like to run in headless mode and handle input differently (for now) - unless anyone has any better ideas about how to disable the rendering without running in headless?

.runsettings doesn't work

It should be possible to tell godot to run in headless mode by passing a parameter in the .runsettings file.

The problem comes: .runsettings does not work in Linux. At least in my experience. Has anyone else noticed this? It doesn't seem to be a problem with this library - I have stepped through the code, and indeed in the ITestDiscoverer, the call to DiscoverTests is passed a DiscoveryContext that has a "garbage" XML with only default values - the actual .runsettings file isn't loaded. I have no idea what to do about this - documentation on the VSTest API seems virtually nonexistent.

Sadly this issue means I couldn't pass --headless to Godot even if I wanted to.

Anyway, there are a few questions/discussion points in there.

[MikeSchulze]

I run the tests on console by this command to specify the .runsettings.
Before you run the test it is important to rebuild the Godot cache otherwise Godot is not able to load the resources.
godot --path ./ -e --headless --quit-after 2000
Run the tests
dotnet test .\Examples\Basics\Tests\Tests.csproj --no-build --settings .runsettings-ci --verbosity normal -m:1

Set the --headless argument in the GdUnit4 parameters section

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <RunConfiguration>
        <MaxCpuCount>1</MaxCpuCount>
        <ResultsDirectory>./TestResults</ResultsDirectory>
        <TargetFramework>net8.0</TargetFramework>
        <TestSessionTimeout>180000</TestSessionTimeout>
        <TreatNoTestsAsError>true</TreatNoTestsAsError>
    </RunConfiguration>

    <LoggerRunSettings>
        <Loggers>
            <Logger friendlyName="console" enabled="True">
                <Configuration>
                    <Verbosity>detailed</Verbosity>
                </Configuration>
            </Logger>
            <Logger friendlyName="html" enabled="True">
                <Configuration>
                    <LogFileName>test-result.html</LogFileName>
                </Configuration>
            </Logger>
            <Logger friendlyName="trx" enabled="True">
                <Configuration>
                    <LogFileName>test-result.trx</LogFileName>
                </Configuration>
            </Logger>
        </Loggers>
    </LoggerRunSettings>

    <GdUnit4>
        <!-- Additonal Godot runtime parameters-->
        <Parameters>--headless</Parameters>
        <!-- Controlls the Display name attribute of the TestCase. Allowed values are SimpleName and FullyQualifiedName.
             This likely determines how the test names are displayed in the test results.-->
        <DisplayName>FullyQualifiedName</DisplayName>
    </GdUnit4>
</RunSettings>

[JustASquid]

OK, thanks! The --settings .runsettings-ci flag totally works. I wonder why it doesn't pick up .runsettings automatically though?

[MikeSchulze]

OK, thanks! The --settings .runsettings-ci flag totally works. I wonder why it doesn't pick up .runsettings automatically though?

I guess it must be placed in the project root.