Skip to content

Commit e33f6f7

Browse files
Disable common order-randomising pytest plugins (#424)
Order-randomising pytest plugins, while brilliant for their intended purpose of rumbling hidden test dependencies, are near-pessimal for mutmut's use case. For a given mutation run, a subset of tests, ordered by increasing runtime, are fed to pytest. The idea is to run fast tests first, increasing the chances of bailing out quickly with a test failure on the mutant being tested. Randomising the order of such tests destroys that ordering and thus blows out mutation test times, sometimes to the point of mutants that would normally be killed ending up timing out instead. As such, pass parms to the pytest runner when testing a mutant to disable two common order-randomisation plugins, pytest-randomly and pytest-random-order.
1 parent d908ad5 commit e33f6f7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

mutmut/__main__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -441,7 +441,7 @@ def pytest_runtest_makereport(self, item, call):
441441
return int(self.execute_pytest(pytest_args, plugins=[stats_collector]))
442442

443443
def run_tests(self, *, mutant_name, tests):
444-
pytest_args = ['-x', '-q']
444+
pytest_args = ['-x', '-q', '-p no:randomly', '-p no:random-order']
445445
if tests:
446446
pytest_args += list(tests)
447447
else:

0 commit comments

Comments
 (0)