Skip to content

Conversation

@LarsAsplund
Copy link
Collaborator

@LarsAsplund LarsAsplund commented May 9, 2025

Added support for test prioritization with two goals in mind:

  • Prioritize likely failures - Tests more likely to fail are executed earlier to provide faster feedback. The following heuristic is used:

      # Test suites are divided into sets which are executed in order. The internal order within a set
      # is decided dynamically at run-time.
      #
      # The 5 sets contains:
      #
      # 0. Test suites that failed before and for which no updates have been made. Expected to fail again.
      # 1. Test suites that failed before but updates have been made that can change that.
      # 2. Test suites without a history. New tests are more likely to fail and should be executed early.
      # 3. Test suites that passed before but depends on updates. There is a risk that we've introduced new bugs.
      # 4. Test suites that passed before and for which there are no updates. They are expected to pass again.
      #
      # Within sets, test suites are sorted in execution time order starting with the fastest test.
      # This is in preparation for the dynamic scheduling that decides the final order within a set.
      # The exception is set 2 which has no history of execution time.
    
  • Load balancing - When tests are executed in parallel using the -p option, VUnit will try to keep the threads balanced. It will initially start with the fastest tests (for fast feedback) but it will also raise the priority of slow tests towards the end of the test run to avoid that some threads complete much later than the others.

Option --changed is also added. It limits tests to those that depend on file changes since the last recorded test run.

@LarsAsplund LarsAsplund merged commit 08b2a6d into master May 9, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants