Skip to content

Change test-execution order (and improve their organization) #1100

Closed
@tcompa

Description

As of #1090, we now have a better organization of tests - including at least subfolders db, api and backend.

We could also add:

  • tests/schemas, with the tests for Pydantic schemas EDIT: this is already in-place
  • tests/unit, with tests of function which do not belong to any other subfolder

Tests in subfolders are typically faster and more low-level than the rest, and they should run first. This will make the CI fail faster for the large fraction of PRs that only change some specific part of the schemas, models or API.

A reasonable order could be like

  1. unit
  2. schemas
  3. db
  4. api
  5. backend (which include SLURM)
  6. all the non-subfolder tests

The ordering feature should be available through https://github.com/pytest-dev/pytest-order (I did not try it out yet).

EDIT: see pytest-dev/pytest-order#52 or pytest-dev/pytest-order#69

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions