Skip to content

fix(simulator_compatibility_test): use pytest runner to fix Jazzy CI failure#390

Merged
xmfcx merged 1 commit intomainfrom
fix/simulator-compatibility-test-jazzy
Apr 2, 2026
Merged

fix(simulator_compatibility_test): use pytest runner to fix Jazzy CI failure#390
xmfcx merged 1 commit intomainfrom
fix/simulator-compatibility-test-jazzy

Conversation

@xmfcx
Copy link
Copy Markdown
Contributor

@xmfcx xmfcx commented Apr 2, 2026

  • Switch tests_require from [] to ["pytest"] so colcon uses the pytest runner instead of unittest
  • Add [tool:pytest] with norecursedirs to prevent pytest from discovering manual simulator tests

Why

The Jazzy CI is failing with exit code 5 on simulator_compatibility_test.

This package is a manual simulator compatibility test suite -- all tests require a running simulator (e.g. MORAI SIM: Drive) and human observation, as described in the README. It has never had automated CI tests, and no tests ran on Humble either:

Starting >>> simulator_compatibility_test
--- output: simulator_compatibility_test
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
---
Finished <<< simulator_compatibility_test [0.62s]

The root cause is a chain of issues:

  1. setup.py had tests_require=[], which made colcon select the unittest runner instead of pytest.
  2. The test classes in this package don't inherit from unittest.TestCase (they're pytest-style), so unittest discovered 0 tests on both distros.
  3. Python 3.10 (Humble) returns exit code 0 for "Ran 0 tests" -- OK.
  4. Python 3.12 (Jazzy) returns exit code 5 for "Ran 0 tests" -- NO TESTS RAN (new behavior).
  5. The colcon unittest runner passes this exit code through directly, failing CI on Jazzy.

The fix switches to the pytest runner by setting tests_require=["pytest"]. The colcon pytest runner explicitly handles exit code 5 (NO_TESTS_COLLECTED) gracefully and does not propagate it as a failure. norecursedirs is needed because pytest has broader test discovery than unittest -- without it, pytest would find and attempt to run the manual simulator tests, which hang waiting for a simulator connection.


Test plan

  • Verify Jazzy CI passes (previously failed with exit code 5)
  • Verify Humble CI still passes (no behavior change -- 0 tests, pass)

…failure

Python 3.12 (Jazzy) changed unittest to return exit code 5 when no
tests are collected, while Python 3.10 (Humble) returns 0. The empty
tests_require caused colcon to use the unittest runner, which never
discovered any tests since the test classes don't inherit
unittest.TestCase. The colcon pytest runner handles exit code 5
gracefully.

Also add norecursedirs to prevent pytest from discovering simulator
tests that require a real MORAI connection.

Signed-off-by: Mete Fatih Cırıt <[email protected]>
@xmfcx xmfcx self-assigned this Apr 2, 2026
@xmfcx xmfcx requested review from shmpwk and tkimura4 as code owners April 2, 2026 09:13
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 2, 2026

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@xmfcx
Copy link
Copy Markdown
Contributor Author

xmfcx commented Apr 2, 2026

The tests have passed, could you review?

@xmfcx xmfcx requested a review from mitsudome-r April 2, 2026 09:23
@xmfcx xmfcx merged commit 5d62188 into main Apr 2, 2026
33 of 34 checks passed
@xmfcx xmfcx deleted the fix/simulator-compatibility-test-jazzy branch April 2, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants