Skip to content

Introduce mutation testing #726

Open
@li-boxuan

Description

We don't have mutation testing used in any coala repository.

As http://pitest.org/ points out,

Traditional test coverage (i.e line, statement, branch, etc.) measures only which code is executed by your tests. It does not check that your tests are actually able to detect faults in the executed code. It is therefore only able to identify code that is definitely not tested.

The most extreme examples of the problem are tests with no assertions. Fortunately these are uncommon in most code bases. Much more common is code that is only partially tested by its suite. A suite that only partially tests code can still execute all its branches (examples).

As it is actually able to detect whether each statement is meaningfully tested, mutation testing is the gold standard against which all other types of coverage are measured.

As far as I am concerned, checkstyle community uses mutation testing, which seems to be quite powerful.

See available mutation testing tools for python: https://github.com/theofidry/awesome-mutation-testing Unfortunately, none of them is very popular, and I have no idea how well they work.

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions