Open
Description
Use cases include:
- alphabetic ordering (or other normalization)
- random order to check that tests are independent of one another
- grouping by "speed": running the tests that are more likely to fail first = decreasing (duration * probability of failure); combined with an option to abort as soon as one test fails (Add an option for early abort #3).
It's not clear how this should be done. One idea is to pass an identifier for the desired order (./test --sort alpha
) on the command line and expect the user code to consult this identifier and sort the test list accordingly (match order with "alpha" -> List.sort ... tests
).