Commit 2bbab42
committed
Disable common order-randomising pytest plugins
Order-randomising pytest plugins, while brilliant for their
intended purpose of rumbling hidden test dependencies, are
near-pessimal for mutmut's use case.
For a given mutation run, a subset of tests, ordered by increasing
runtime, are fed to pytest. The idea is to run fast tests first,
increasing the chances of bailing out quickly with a test failure
on the mutant being tested.
Randomising the order of such tests destroys that ordering and
thus blows out mutation test times, sometimes to the point of
mutants that would normally be killed ending up timing out
instead.
As such, pass parms to the pytest runner when testing a mutant
to disable two common order-randomisation plugins,
pytest-randomly and pytest-random-order.1 parent d908ad5 commit 2bbab42
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
441 | 441 | | |
442 | 442 | | |
443 | 443 | | |
444 | | - | |
| 444 | + | |
445 | 445 | | |
446 | 446 | | |
447 | 447 | | |
| |||
0 commit comments