xdist currently supports 2 ways of distributing tests across a number of workers:

1. "Each" Scheduling: Given a set of tests, "each" scheduling sends each test in the set to each available worker.
2. Load Scheduling: Given a set of tests, load scheduling distributes the tests among the available workers. Each test runs only once, and xdist tries to give each worker the same amount of work.

Problems with Load Scheduling's current implementation

The current load scheduling implementation distributes tests naively across the workers. Often this means that two tests which depend on the same fixture get assigned to different runners, and the fixture has to be created on each runner.

This is a problem. Fixtures often capture expensive operations. When multiple tests depend on the same fixture, the author typically expects the expensive operation represented by that fixture to happen only once and be reused by all dependent tests. When tests that depend on the same fixture are sent to different workers, that expensive operation is executed multiple times. This is wasteful and can add significantly to the overall testing runtime.

Our goal should be to reuse the existing pytest concept of the fixture to better distribute tests and reduce overall testing time, preferably without adding new options or APIs. This benefits the most users and strengthens pytest's declarative style.

Proposed solution

We can solve this problem in 3 phases:

1. Let's formalize the concept of a "test chunk" or "test block". A test chunk is a group of tests that always execute on the same worker. This is an internal xdist abstraction that the user normally doesn't have to know about.

   The master will only send complete test chunks to workers, not individual tests. Initially, each test will be assigned to its own chunk, so this won't change xdist's behavior at first. But it will pave the way for us to chunk tests by their attributes, like what fixtures they depend on.

   Once we have this internal abstraction, we can optionally also expose a hook that lets users define their own chunking algorithm to replace the initial default of "1 test -> 1 chunk". The hook won't be very useful until more information about each test is made available, which brings us to the next phase.

2. We need to pass in additional information to the xdist master about the tests it's running so it can better distribute them. Specifically the master needs to be able to identify unique instances of every fixture that each test depends on. Tests that depend on distinct fixture instances can be assigned to different chunks and thus sent to different workers.

   To identify the distinct fixture instances that each test depends on, we need the following pieces of information for each test:

   1. Test name.
   2. For each fixture that the test depends on:
      1. Fixture name.
      2. Fixture scope and "scope instance".
         • e.g. This fixture is module-scoped and this particular instance of the fixture is for module test_a.py.
      3. Fixture parameter inputs, if the fixture is parameterized.
         • e.g. We need to distinguish fixture(request.param=1) and fixture(request.param=2) as separate fixture instances.

   Initially this information won't be used for anything. It will just be made available to the master. At this point, we'll be ready for the final phase.

3. Using the new information we have about tests, along with the new internal abstraction of a test chunk, we can now chunk tests up by the list of unique fixture instances they depend on.

   Tests that depend on the same instance of a fixture will now always be sent to the same worker. 🎉

Advantages over other solutions

This approach has two major advantages over other proposals.

1. This approach adds no new configuration options or public APIs that users have to learn. Everyone using pytest automatically gets a better, arguably even more correct, distribution of their tests to workers without having to do any work.
2. This approach promotes a declarative style of writing tests over an imperative one. The goal we should strive for is: Capture your ideas correctly, and pytest will figure out the appropriate execution details. In practice, this is probably not always feasible, and the user will want to exercise control in specific cases. But the closer we can stick to this goal the better the pytest user experience will be.

How this approach addresses common use cases

There are several use cases described in the original issue requesting some way to control parallel execution. This is how the approach described here addresses those use cases:

• Use case: "I want all the tests in a module to go to the same worker."

  Solution: Make all the tests in that module depend on the same module-scoped fixture.

  If the tests don't need to depend on the same fixture, then why do they need to go to the same worker? (The most likely reason is that they are not correctly isolated.)

• Use case: "I want all the tests in a class to go the same worker."

  Solution: The solution here is the same. If the tests need to go to the same worker, then they should depend on a common fixture.

  To cleanly address this case, we may need to implement the concept of a class-scoped fixture. (Fixtures can currently be scoped only by function, module, or session.) Otherwise, the tests can depend on a common, module-scoped fixture and achieve the same result.

• Use case: "I want all the tests in X to go to the same worker."

  Solution: You know the drill. If tests belong on the same worker, we are betting that there is an implied, shared dependency. Express that shared dependency as a fixture and pytest will take care of the rest for you.

Counter-proposal to #17.
Addresses pytest-dev/pytest#175.
Adapted from this comment by @hpk42.