Skipped test should be forbidden #750
Description
Description
In #745, the configuration to forbid only and pending tests on CI was added. This successfully prevent the use of .only on CI. But actually does not forbid .skip as seen in the CI log of #745: https://github.com/jupyterlab/lumino/actions/runs/13718590093/job/38368769283
Relevant CI log for posterity
> @lumino/widgets:"test:chromium"
@lumino/[email protected] test
npm run build:test && web-test-runner tests/lib/bundle.test.js --node-resolve --playwright --browsers chromium
@lumino/[email protected] build:test
npm run clean:test && tsc --build tests && cd tests && rollup -c
@lumino/[email protected] clean:test
rimraf tests/lib tests/tsconfig.tsbuildinfo
./lib/index.spec.js → ./lib/bundle.test.js...
(!) Circular dependencies
../../../node_modules/chai/lib/chai.js -> ../../../node_modules/chai/lib/chai/utils/index.js -> ../../../node_modules/chai/lib/chai/utils/addProperty.js -> ../../../node_modules/chai/lib/chai.js
../../../node_modules/chai/lib/chai.js -> ../../../node_modules/chai/lib/chai/utils/index.js -> ../../../node_modules/chai/lib/chai/utils/addMethod.js -> ../../../node_modules/chai/lib/chai.js
../../../node_modules/chai/lib/chai.js -> ../../../node_modules/chai/lib/chai/utils/index.js -> ../../../node_modules/chai/lib/chai/utils/overwriteProperty.js -> ../../../node_modules/chai/lib/chai.js
...and 3 more
created ./lib/bundle.test.js in 1.9s
Chromium: |██████████████████████████████| 0/1 test files | 0 passed, 0 failed
Running tests...
Running 1 test files...
Chromium: |██████████████████████████████| 1/1 test files | 938 passed, 0 failed, 1 skipped
Reproduce
- Flag a test with
it.skip - Set environment variable
CI=1 - Execute the tests, it does not fail
Expected behavior
When on CI, skipped tests should result in failing tests.
Important
The test do failed when run in debug mode within the browser.