Skip to content

way to escalate failure when comparing outside of pytest #273

Open
@nico-ssi

Description

compare outside of pytest allows to compare 2 previous benchmark runs.

However, there is no way to do --benchmark-compare-fail outside of pytest.

Please consider providing a flag i.e.

pytest-benchmark compare 0001 0002 --fail=<stat>:<num>

Then return a non-zero exit status when the fail criteria is met.

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions