Performance regression

[tomsterBG]

Versions

(list all versions where you have replicated the bug)

• Godot: 4.7.1
• GUT: 9.4.0 -> 9.7.1

The Bug

GUT 9.4.0 does all my tests at https://github.com/tomsterBG/short in 0.08 to 0.09 seconds.
GUT 9.7.1 does all my tests in 0.36 to 0.38 seconds.

Steps To Reproduce

Have a test suite with upwards of 1000 asserts.
Use GUT 9.4.0 and GUT 9.7.1 on the same suite.

More info

While the jump from 0.09 to 0.36 is not big, it's a 4x difference and the jump from 0.08 to 0.38 is a 4.75x difference. You can easily imagine how this can ruin large scale test suites that have way more than 1000 asserts including performance related tests (like how i have a test_all.gd to ensure damaging a health doesn't take a stupidly long amount of time, which is expected to take less than 0.02s for a health with 1016 shields).

If you try to use my project to test for yourself, you may have errors as i have uncommitted local changes that i need to sort out.

[bitwes]

This is due to an increase in post-run cleanup time. You can use the attached project to see that teardown time has increased from 19 msecs to 428 msecs but the execution time on the included test scripts hasn't changed.

The project has some tests and uses hook scripts to print out the time in msecs between different parts of the test run. All the output from the hook scripts starts with -> .

exec_time_project.zip

In the below examples the gut command is:

<godot> -s addons/gut/gut_cmdln.gd

Godot 4.4.1 / GUT 9.4.0

Godot Engine v4.4.1.stable.official.49a5bc7b6 - https://godotengine.org
Metal 3.2 - Forward+ - Using Device #0: Apple - Apple M1 Pro (Apple7)

---  GUT  ---
[INFO]:  using [/Users/butchwesley/Library/Application Support/Godot/app_userdata/[unnamed project]] for temporary output.
Godot version:  4.4.1
GUT version:  9.4.0
-> START 793


res://test/test_something.gd
-> Startup 0
* test_something
1/1 passed.
-> Script duration:  0


res://test/test_something_else.gd
-> Startup 0
* test_something
* test_assert_one_thousand_times
* test_assert_more_thousands_times
3/3 passed.
-> Script duration:  76




==============================================
= Run Summary
==============================================

---- Totals ----
Scripts           2
Tests             4
  Passing         4
Asserts           6002
Time              0.095s


---- All tests passed! ----

-> END 888
-> Teardown: 19
-> Duration 95

Godot 4.7.1 / GUT 9.7.1

➜  godot_4_7 gut
Godot Engine v4.7.1.stable.official.a13da4feb - https://godotengine.org
Metal 4.0 - Forward+ - Using Device #0: Apple - Apple M1 Pro (Apple7)

---  GUT  ---
[INFO]:  using [/Users/butchwesley/Library/Application Support/Godot/app_userdata/[unnamed project]] for temporary output.
Godot version:  4.7.1
GUT version:  9.7.1
-> START 814


res://test/test_something.gd
-> Startup 0
* test_something
1/1 passed.
-> Script duration:  0


res://test/test_something_else.gd
-> Startup 0
* test_something
* test_assert_one_thousand_times
* test_assert_more_thousands_times
3/3 passed.
-> Script duration:  69
-> END 1311
-> Teardown: 428
-> Duration 497




==============================================
= Run Summary
==============================================

Totals
------
Scripts               2
Tests                 4
Passing Tests         4
Asserts            6002
Time              0.498s


---- All tests passed! ----

➜  godot_4_7