Skip to content

Comments

WIP: Allow cosmos tests to run in parallel v2#37766

Draft
JoasE wants to merge 9 commits intodotnet:mainfrom
JoasE:feature/cosmos-test-paralellism-v2
Draft

WIP: Allow cosmos tests to run in parallel v2#37766
JoasE wants to merge 9 commits intodotnet:mainfrom
JoasE:feature/cosmos-test-paralellism-v2

Conversation

@JoasE
Copy link
Contributor

@JoasE JoasE commented Feb 21, 2026

With some spare time, I've been experimenting around with the cosmos functional tests to see if I can get them to run in parallel.

The biggest challenge appeared to be the cosmos db emulator, which can become unstable in certain scenario's.
After some testing I found that the create container implementation in the emulator has a race condition, and can crash the emulator when ran in parallel. Other CRUD requests for containers appear to have no direct effect on the running of the emulator, but parallel requests can cause the emulator to have to clear the data upon restart.
In addition, the docs state that the emulator could show performance degradation with more than 10 concurrent containers.
On top of that, my tests have shown the emulator will stop responding to create container requests at some point when creating many containers. On my local machine this happens consistently after the creation of the 25th container.

This means that when running the tests against the emulator we don't execute CRUD container requests in parallel and ideally no more than 10 containers exist at the same time, and never more than 25.

This could (currently) be fixed by just decreasing the parallelism using CollectionBehavior(MaxParallelThreads = N), but this is risky as it will cause errors or slow tests when many tests are added that create a lot of containers and those happen to run in parallel. See: #37696

Another option would be to manually control parallelism in CosmosTestStore based on the number of containers each test store is attempting to create. This is both faster and prevents possible errors, but there are a couple of downsides, being mainly:

  • Complex-ish logic simply for running tests faster, albeit only scoped to tests running against the emulator
  • Test execution time of non-shared model tests are affected by the wait time introduced by the manual parallelism control.

This pr implements the latter.

Some other challenges were:

  • Different test classes might use the same shared test fixture, so each tests needs a unique store or we have to keep track of which stores are being used by which tests so we know when we can delete them.
  • Some non-shared model tests use the same test store, as they inherit from other non-shared model tests. These tests can't always be ran in parallel against the same store, so they need a unique store.
  • Some tests create containers outside of the test store, which could cause parallel create container requests

There was a custom import for a cosmos specific Northwind database, which didn't appear to be needed. The custom import was very slow compared to using savechange's new batching, so I removed it and had to slightly change 1 test.

This PR decreases total test run time:
locally:
from 26 to 12.5 minutes
pipeline: TODO

Run against real cosmos db: TODO

@JoasE JoasE changed the title Allow cosmos tests to run in parallel v2 WIP: Allow cosmos tests to run in parallel v2 Feb 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant