This was generated by AI during triage.
Summary
The CI workflow (.github/workflows/ci.yml) is failing on both push and pull_request events on frontend-game-mockup. The failure is in step 7 — Run gameworks full test suite (headless) on both Python 3.11 and Python 3.12 matrix slots. All earlier steps (checkout, setup-python, install deps, pipeline tests, engine tests) pass cleanly.
Failing Runs
| Run ID |
Event |
Commit SHA |
Run # |
Duration |
| 26494894352 |
push |
28bc212f |
CI #162 |
11m 12s |
| 26494896294 |
pull_request (PR #13) |
28bc212f |
CI #163 |
10m 26s |
Both runs failed on the same commit (28bc212fce3fcb6630cbddb64d58c7067fb83b40, titled "fix(ci): resolve all render-perf workflow failures"). The push run and PR run are duplicates — same code, same result.
Failing Step Detail
Step 7 — Run gameworks full test suite (headless) — conclusion: failure
- name: Run gameworks full test suite (headless)
env:
SDL_AUDIODRIVER: dummy
run: |
python -m pytest gameworks/tests/ \
--ignore=gameworks/tests/performance/test_property_based.py -vResults (Python 3.11): 7 failed, 784 passed, 17 skipped in 538.89s (08:58)
Results (Python 3.12): 7 failed, 784 passed, 17 skipped in ~10:28
All 7 Failing Tests
FAIL-1 — test_mutation_targets.py::TestG4MutantKillers::test_has_flash_short_circuit_is_bool_check
AssertionError: GC fired 15 times during 100 frames with no mine flash.
The _has_flash short-circuit is not eliminating dict lookups.
assert 15 == 0
What the test asserts: With no active mine flashes (_has_flash = False), the renderer's draw path must short-circuit and skip all dict lookups in the flash-drawing code. This should produce zero gen-0 GC collections across 100 rendered frames.
Root cause: The _has_flash boolean check in gameworks/renderer.py is either absent, placed off the hot path, or — more likely — even when False, the surrounding draw code still performs allocations that produce GC-collectable objects. 15 collections over 100 frames = roughly one GC every 6-7 frames regardless of flash state.
FAIL-2 — test_properties.py::test_prop_iter190_frame_simple_in_draw_board
AssertionError: _frame_simple not present in renderer
assert '_frame_simple' in '<full source text of gameworks/renderer.py>'
Falsifying example: test_prop_iter190_frame_simple_in_draw_board(board_w=40)
What the test asserts: This is a Hypothesis-driven source inspection test — it reads the full text of gameworks/renderer.py and asserts that the string _frame_simple appears somewhere in the source. This enforces that a private optimisation attribute/method named _frame_simple is physically present in the renderer implementation.
Root cause: _frame_simple does not exist anywhere in gameworks/renderer.py. The Hypothesis board_w=40 in the falsifying example is a red herring — the failure occurs on every single board width because it is checking the static source text. Either:
_frame_simple was removed during a renderer refactor and this test was not updated, or- This test was written speculatively for a planned implementation that was never committed.
FAIL-3 through FAIL-7 — test_zoom_latency.py::test_g4_zero_gc_during_steady_state (5 parametrized board sizes)
| Parametrize ID |
Board Dimensions |
gen-0 collections |
Expected |
Guardrail |
[board_wh0-SMALL] |
small |
13 |
0 |
G4-ELITE VIOLATION |
[board_wh1-MEDIUM] |
medium |
13 |
0 |
G4-ELITE VIOLATION |
[board_wh2-LARGE] |
large |
13 |
0 |
G4-ELITE VIOLATION |
[board_wh3-XLARGE] |
1500x1080 |
13 |
0 |
G4-ELITE VIOLATION |
[board_wh4-MEGA] |
1920x1080 |
13 |
0 |
G4-ELITE VIOLATION |
Full assertion from test output:
AssertionError: G4-ELITE VIOLATION — SMALL board: 13 gen-0 GC collections
during 500 steady-state frames. Expected: 0.
Run tracemalloc to identify the allocation site.
assert 13 == 0
gc_data = {'frames': 500, 'gen0_collections': 13, 'gen1_collections': 0, 'gen2_collections': 0}
What the test asserts: The G4-ELITE guardrail requires that after the renderer reaches steady state (board initialised, game running, no user input), executing 500 render frames produces exactly zero gen-0 GC collections. The renderer's hot path must not allocate any tracked Python objects during steady state.
Root cause: gameworks/renderer.py allocates tracked Python objects on its hot draw path during steady-state rendering. The identical count of 13 across all 5 board sizes (SMALL through MEGA) is the most diagnostic clue: the allocations are board-size-independent, pointing to a fixed set of per-frame object creations — likely one or more of: list/dict/tuple literals constructed inside a loop, closure captures, Pygame Surface creation, or intermediate str.format()/f-string calls in the draw path. The test instructs running tracemalloc to find the allocation site.
Note: The _zoom_results_v2 dict in the assertion context is fully populated with passing G1 results — the G1 latency guardrail is not violated. Only the G4 zero-GC guardrail is broken.
Performance Regression Report (from render-perf companion run)
The check_perf_regression.py script ran successfully after the benchmark step (via if: always()) and reported all metrics within threshold — confirming the latency/timing guardrails (G1, G2, G5) are passing. The only failures are the GC-related and structural tests above.
Metric Threshold Actual Ratio Status
g1_handle_event_SMALL 0.40s 0.006s 0.01x PASS
g2_first_draw_no_image_MEGA 10.00s 8.576s 0.86x WARN
g2_first_draw_image_MEGA 18.00s 10.731s 0.60x PASS
g5_e2e_latency_MEGA 16.00s 13.409s 0.84x PASS
... (all 20 metrics pass or warn)
All metrics within threshold.
History of Failed Fix Attempts (2026-05-26 / 2026-05-27)
Three commits were pushed to frontend-game-mockup on 2026-05-27 in an attempt to fix CI. None of them fixed the 7 test failures that now block CI.
Attempt 1 — 4707d12d at 2026-05-27T04:29:50Z
Commit title: fix(deps): pin contourpy and tifffile to versions compatible with Python 3.10-3.12
Changes made:
contourpy: 1.3.3 → 1.3.2 (last version with Python 3.10 wheels)tifffile: 2026.5.15 → 2026.3.3 (last version before Requires-Python>=3.12)
Reasoning behind the attempt: The CI matrix included Python 3.10 at the time. contourpy==1.3.3 requires Python>=3.11 and tifffile==2026.5.15 requires Python>=3.12. Both caused pip to abort the entire dependency resolution before installing anything — meaning pytest was never installed and all test steps failed with command not found or similar. The diagnosis was correct: pip was crashing before tests could run. The fix was to pin these two packages to older versions that have Python 3.10 wheels.
Flawed logic: The dep pinning was a correct and necessary fix for the pip crash, but it was treated as the complete solution. Once pip succeeded, the 7 underlying test failures became visible for the first time. The commit assumed tests would pass once the environment was functional — they did not.
Outcome: Pip install now succeeds. CI runs tests. 7 tests fail.
Attempt 2 — d475bb15 at 2026-05-27T04:38:24Z
Commit title: fix(ci): drop Python 3.10, fix pyflakes errors in perf tests
Changes made:
- Removed Python 3.10 from the CI matrix (now only 3.11 and 3.12)
- Fixed unused imports/NameErrors in performance test files:
conftest.py: removed unused sys, typing.Tupletest_property_based.py: removed unused assume; fixed NameError — width/height were undefined (should be board_w/board_h)test_zoom_latency.py: removed unused tracemalloctest_mutation_targets.py: removed unused time and bare import pygame
Reasoning behind the attempt: Rather than continuing to fight dep compatibility for 3.10, the approach was to narrow the CI matrix to match the actual supported Python range (3.11+). The pyflakes errors were caught by the pyflakes step in render-perf.yml and needed fixing regardless. The assumption was: fix the environment issues + fix the import errors = green CI.
Flawed logic: Import errors and matrix narrowing are orthogonal to the 7 failing tests:
test_g4_zero_gc_during_steady_state fails because the renderer allocates objects — not because of an unused tracemalloc import.test_prop_iter190_frame_simple_in_draw_board fails because _frame_simple is not in the renderer source — not because of a NameError in an unrelated file.test_has_flash_short_circuit_is_bool_check fails because GC fires despite _has_flash=False — not because of an unused time import.
Removing the unused tracemalloc import from test_zoom_latency.py was a pyflakes fix, but completely unrelated to the GC test logic.
Outcome: Cleaner imports, narrower matrix. Same 7 test failures.
Attempt 3 — 28bc212f at 2026-05-27T06:31:08Z — THE COMMIT THAT TRIGGERED THESE RUNS
Commit title: fix(ci): resolve all render-perf workflow failures
Changes made:
- Added
--ignore=gameworks/tests/performance/test_property_based.py to both render-perf.yml and ci.yml pytest invocations
- Fixed filename mismatch:
test_zoom_latency.py was writing results to zoom_latency_results_v3.json but the workflow and check_perf_regression.py expected render_perf_results.json
- Created
perf-results/baseline.json — check_perf_regression.py opened this file unconditionally on startup; the file was missing from the repo entirely, causing a crash before any comparison
- Fixed
check_perf_regression.py: extracted p95_ms/p95_total_ms from nested dict results before computing ratio — the previous code attempted to divide a dict object by a float
Reasoning behind the attempt: The commit message claims to "resolve ALL render-perf workflow failures." The reasoning was:
test_property_based.py uses Hypothesis which fails on CI shared runners due to G1-FLOOR violations and timeout during shrinking — correct to exclude- The filename mismatch meant
render_perf_results.json was never produced — correct fix
- Missing
baseline.json caused check_perf_regression.py to crash with FileNotFoundError — correct fix
check_perf_regression.py dividing a dict by a float was a TypeError — correct fix
Flawed logic — critical: All four fixes in this commit addressed CI infrastructure problems. They were all correct. But the commit message's claim to resolve "ALL" failures was wrong. After excluding test_property_based.py, three categories of genuine test failures remained unaddressed:
-
G4-ELITE GC violations (test_g4_zero_gc_during_steady_state x 5): These are real product bugs — the renderer allocates memory on its hot path. The author (Claude Sonnet 4.6) did not investigate what was actually failing inside test_zoom_latency.py before shipping; it was assumed that excluding test_property_based.py (the Hypothesis file) would clear the performance test suite.
-
Missing _frame_simple attribute (test_prop_iter190_frame_simple_in_draw_board): This test was in test_properties.py, not test_property_based.py. The exclusion of the _based file did not exclude this test. The author did not check which tests in test_properties.py would still run after the exclusion.
-
_has_flash short-circuit absent/broken (test_has_flash_short_circuit_is_bool_check): This is in test_mutation_targets.py, which was also not excluded. The mutation-testing framework requires that specific optimisation code actually exist in the renderer — it cannot be satisfied by CI configuration changes.
Root misdiagnosis: The entire series of 3 fix commits treated CI failures as an environment/configuration problem solvable without touching gameworks/renderer.py. The actual failures require code changes in the renderer itself.
Outcome: Infrastructure issues fixed. 7 test failures remain. These 2 CI runs (plus 2 render-perf runs) are the proof.
Acceptance Criteria
Blocked by
None — can start immediately.
Related
The same 7 tests also fail in the Render Performance Smoke workflow — see companion issue for runs #26494894350 and #26494896365.
Summary
The CI workflow (
.github/workflows/ci.yml) is failing on both push and pull_request events onfrontend-game-mockup. The failure is in step 7 —Run gameworks full test suite (headless)on both Python 3.11 and Python 3.12 matrix slots. All earlier steps (checkout, setup-python, install deps, pipeline tests, engine tests) pass cleanly.Failing Runs
push28bc212fpull_request(PR #13)28bc212fBoth runs failed on the same commit (
28bc212fce3fcb6630cbddb64d58c7067fb83b40, titled "fix(ci): resolve all render-perf workflow failures"). The push run and PR run are duplicates — same code, same result.Failing Step Detail
Step 7 — Run gameworks full test suite (headless) —
conclusion: failureResults (Python 3.11): 7 failed, 784 passed, 17 skipped in 538.89s (08:58)
Results (Python 3.12): 7 failed, 784 passed, 17 skipped in ~10:28
All 7 Failing Tests
FAIL-1 —
test_mutation_targets.py::TestG4MutantKillers::test_has_flash_short_circuit_is_bool_checkWhat the test asserts: With no active mine flashes (
_has_flash = False), the renderer's draw path must short-circuit and skip all dict lookups in the flash-drawing code. This should produce zero gen-0 GC collections across 100 rendered frames.Root cause: The
_has_flashboolean check ingameworks/renderer.pyis either absent, placed off the hot path, or — more likely — even whenFalse, the surrounding draw code still performs allocations that produce GC-collectable objects. 15 collections over 100 frames = roughly one GC every 6-7 frames regardless of flash state.FAIL-2 —
test_properties.py::test_prop_iter190_frame_simple_in_draw_boardWhat the test asserts: This is a Hypothesis-driven source inspection test — it reads the full text of
gameworks/renderer.pyand asserts that the string_frame_simpleappears somewhere in the source. This enforces that a private optimisation attribute/method named_frame_simpleis physically present in the renderer implementation.Root cause:
_frame_simpledoes not exist anywhere ingameworks/renderer.py. The Hypothesisboard_w=40in the falsifying example is a red herring — the failure occurs on every single board width because it is checking the static source text. Either:_frame_simplewas removed during a renderer refactor and this test was not updated, orFAIL-3 through FAIL-7 —
test_zoom_latency.py::test_g4_zero_gc_during_steady_state(5 parametrized board sizes)[board_wh0-SMALL][board_wh1-MEDIUM][board_wh2-LARGE][board_wh3-XLARGE][board_wh4-MEGA]Full assertion from test output:
What the test asserts: The G4-ELITE guardrail requires that after the renderer reaches steady state (board initialised, game running, no user input), executing 500 render frames produces exactly zero gen-0 GC collections. The renderer's hot path must not allocate any tracked Python objects during steady state.
Root cause:
gameworks/renderer.pyallocates tracked Python objects on its hot draw path during steady-state rendering. The identical count of 13 across all 5 board sizes (SMALL through MEGA) is the most diagnostic clue: the allocations are board-size-independent, pointing to a fixed set of per-frame object creations — likely one or more of: list/dict/tuple literals constructed inside a loop, closure captures, Pygame Surface creation, or intermediatestr.format()/f-string calls in the draw path. The test instructs runningtracemallocto find the allocation site.Note: The
_zoom_results_v2dict in the assertion context is fully populated with passing G1 results — the G1 latency guardrail is not violated. Only the G4 zero-GC guardrail is broken.Performance Regression Report (from render-perf companion run)
The
check_perf_regression.pyscript ran successfully after the benchmark step (viaif: always()) and reported all metrics within threshold — confirming the latency/timing guardrails (G1, G2, G5) are passing. The only failures are the GC-related and structural tests above.History of Failed Fix Attempts (2026-05-26 / 2026-05-27)
Three commits were pushed to
frontend-game-mockupon 2026-05-27 in an attempt to fix CI. None of them fixed the 7 test failures that now block CI.Attempt 1 —
4707d12dat 2026-05-27T04:29:50ZCommit title:
fix(deps): pin contourpy and tifffile to versions compatible with Python 3.10-3.12Changes made:
contourpy: 1.3.3 → 1.3.2 (last version with Python 3.10 wheels)tifffile: 2026.5.15 → 2026.3.3 (last version beforeRequires-Python>=3.12)Reasoning behind the attempt: The CI matrix included Python 3.10 at the time.
contourpy==1.3.3requires Python>=3.11 andtifffile==2026.5.15requires Python>=3.12. Both causedpipto abort the entire dependency resolution before installing anything — meaningpytestwas never installed and all test steps failed withcommand not foundor similar. The diagnosis was correct: pip was crashing before tests could run. The fix was to pin these two packages to older versions that have Python 3.10 wheels.Flawed logic: The dep pinning was a correct and necessary fix for the pip crash, but it was treated as the complete solution. Once pip succeeded, the 7 underlying test failures became visible for the first time. The commit assumed tests would pass once the environment was functional — they did not.
Outcome: Pip install now succeeds. CI runs tests. 7 tests fail.
Attempt 2 —
d475bb15at 2026-05-27T04:38:24ZCommit title:
fix(ci): drop Python 3.10, fix pyflakes errors in perf testsChanges made:
conftest.py: removed unusedsys,typing.Tupletest_property_based.py: removed unusedassume; fixed NameError —width/heightwere undefined (should beboard_w/board_h)test_zoom_latency.py: removed unusedtracemalloctest_mutation_targets.py: removed unusedtimeand bareimport pygameReasoning behind the attempt: Rather than continuing to fight dep compatibility for 3.10, the approach was to narrow the CI matrix to match the actual supported Python range (3.11+). The pyflakes errors were caught by the
pyflakesstep inrender-perf.ymland needed fixing regardless. The assumption was: fix the environment issues + fix the import errors = green CI.Flawed logic: Import errors and matrix narrowing are orthogonal to the 7 failing tests:
test_g4_zero_gc_during_steady_statefails because the renderer allocates objects — not because of an unusedtracemallocimport.test_prop_iter190_frame_simple_in_draw_boardfails because_frame_simpleis not in the renderer source — not because of a NameError in an unrelated file.test_has_flash_short_circuit_is_bool_checkfails because GC fires despite_has_flash=False— not because of an unusedtimeimport.Removing the unused
tracemallocimport fromtest_zoom_latency.pywas a pyflakes fix, but completely unrelated to the GC test logic.Outcome: Cleaner imports, narrower matrix. Same 7 test failures.
Attempt 3 —
28bc212fat 2026-05-27T06:31:08Z — THE COMMIT THAT TRIGGERED THESE RUNSCommit title:
fix(ci): resolve all render-perf workflow failuresChanges made:
--ignore=gameworks/tests/performance/test_property_based.pyto bothrender-perf.ymlandci.ymlpytest invocationstest_zoom_latency.pywas writing results tozoom_latency_results_v3.jsonbut the workflow andcheck_perf_regression.pyexpectedrender_perf_results.jsonperf-results/baseline.json—check_perf_regression.pyopened this file unconditionally on startup; the file was missing from the repo entirely, causing a crash before any comparisoncheck_perf_regression.py: extractedp95_ms/p95_total_msfrom nested dict results before computing ratio — the previous code attempted to divide a dict object by a floatReasoning behind the attempt: The commit message claims to "resolve ALL render-perf workflow failures." The reasoning was:
test_property_based.pyuses Hypothesis which fails on CI shared runners due to G1-FLOOR violations and timeout during shrinking — correct to excluderender_perf_results.jsonwas never produced — correct fixbaseline.jsoncausedcheck_perf_regression.pyto crash withFileNotFoundError— correct fixcheck_perf_regression.pydividing a dict by a float was a TypeError — correct fixFlawed logic — critical: All four fixes in this commit addressed CI infrastructure problems. They were all correct. But the commit message's claim to resolve "ALL" failures was wrong. After excluding
test_property_based.py, three categories of genuine test failures remained unaddressed:G4-ELITE GC violations (
test_g4_zero_gc_during_steady_statex 5): These are real product bugs — the renderer allocates memory on its hot path. The author (Claude Sonnet 4.6) did not investigate what was actually failing insidetest_zoom_latency.pybefore shipping; it was assumed that excludingtest_property_based.py(the Hypothesis file) would clear the performance test suite.Missing
_frame_simpleattribute (test_prop_iter190_frame_simple_in_draw_board): This test was intest_properties.py, nottest_property_based.py. The exclusion of the_basedfile did not exclude this test. The author did not check which tests intest_properties.pywould still run after the exclusion._has_flashshort-circuit absent/broken (test_has_flash_short_circuit_is_bool_check): This is intest_mutation_targets.py, which was also not excluded. The mutation-testing framework requires that specific optimisation code actually exist in the renderer — it cannot be satisfied by CI configuration changes.Root misdiagnosis: The entire series of 3 fix commits treated CI failures as an environment/configuration problem solvable without touching
gameworks/renderer.py. The actual failures require code changes in the renderer itself.Outcome: Infrastructure issues fixed. 7 test failures remain. These 2 CI runs (plus 2 render-perf runs) are the proof.
Acceptance Criteria
test_g4_zero_gc_during_steady_state[board_wh0-SMALL]through[board_wh4-MEGA]all passtest_has_flash_short_circuit_is_bool_checkpasses —_has_flash=Falseeliminates GC in 100 framestest_prop_iter190_frame_simple_in_draw_boardeither passes (implement_frame_simple) or is deleted with documented justification that it is a stale structural assertionBlocked by
None — can start immediately.
Related
The same 7 tests also fail in the Render Performance Smoke workflow — see companion issue for runs #26494894350 and #26494896365.