Open
Description
compare outside of pytest allows to compare 2 previous benchmark runs.
However, there is no way to do --benchmark-compare-fail
outside of pytest.
Please consider providing a flag i.e.
pytest-benchmark compare 0001 0002 --fail=<stat>:<num>
Then return a non-zero exit status when the fail criteria is met.
Metadata
Assignees
Labels
No labels