Change test-execution order (and improve their organization) #1100
Closed
Description
As of #1090, we now have a better organization of tests - including at least subfolders db
, api
and backend
.
We could also add:
EDIT: this is already in-placetests/schemas
, with the tests for Pydantic schemastests/unit
, with tests of function which do not belong to any other subfolder
Tests in subfolders are typically faster and more low-level than the rest, and they should run first. This will make the CI fail faster for the large fraction of PRs that only change some specific part of the schemas, models or API.
A reasonable order could be like
- unit
- schemas
- db
- api
- backend (which include SLURM)
- all the non-subfolder tests
The ordering feature should be available through https://github.com/pytest-dev/pytest-order (I did not try it out yet).
EDIT: see pytest-dev/pytest-order#52 or pytest-dev/pytest-order#69