Fail loudly instead of skipping cea_module fixture on broken CI builds - #151
Open
djkees wants to merge 1 commit into
Open
Fail loudly instead of skipping cea_module fixture on broken CI builds#151djkees wants to merge 1 commit into
djkees wants to merge 1 commit into
Conversation
#208) In CI, import/init failure in the fixture now calls pytest.fail instead of pytest.skip, so a broken build can't report as a green, all-skipped run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The session-scoped
cea_modulepytest fixture insource/bind/python/tests/conftest.py, which nearly every Python test depends on, calledpytest.skip(...)whenimport ceaorcea.is_initialized()failed. A broken build (compile failure, missing extension, botched CMake step) made every dependent test report "skipped" rather than "failed," and pytest exits 0 on an all-skipped run — sobasic_build.yml'spython-testsjob could show green even though nothing actually ran.djkees#89
Changes
conftest.py: the fixture now checks theCI/GITHUB_ACTIONSenvironment variables (set automatically by GitHub Actions runners). When set, an import or init failure callspytest.fail(...)instead ofpytest.skip(...), so the run errors out with a nonzero exit code. Outside CI (e.g. local dev without a built extension), the original skip behavior is unchanged.Testing
pytest source/bind/python/tests -q(real dev environment, extension built): 116 passed, confirming no regression.sys.modules["cea"]to force an import failure: withoutCIset,1 skipped(unchanged); withCI=true,1 error/Failed: cea import failed: ...(new behavior).Compatibility / Numerical behavior
Test-infrastructure only; no solver/binding logic touched.
Drafted with Claude's assistance
conftest.pyand reproducing both the old (skip) and new (fail) behavior directly, in both a clean sandbox and a real dev environment, before and after the change.