Integration: Run tests concurrently #326
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today all of the tests are ran serially which on an M1 Max comes out to usually around 30+ seconds for me. This seems much too long to wait for what these tests actually do.. There was one decent deterrent in the way of getting these to actually run concurrently however, which is that today every test uses the exact same image, and it is unpacked every single time in the `bootstrap` method. Every invocation after the first doesn't actually re-unpack the image, as the logic for "is this unpacked" is just a stat on the path, but what this means once we try and run the same on > 1 thread is they will all compete with each other on unpacking, and sometimes (almost always) trample over each other. This can result in guests trying to mount a faulty filesystem: [ 0.303404] EXT4-fs (vdb): VFS: Can't find ext4 filesystem Ideally we have API to handle this, but for now I've invented a shoddy unpack coordinator type that just keys off of a path and halts any other threads if an unpack for that path is already in progress. Timings (10 runs, image already pulled, max concurrency == 8): New avg: 11s Old Avg: 31s
dcantah
force-pushed
the
integration-tests-concurrent
branch
from
6369a8a to
f75092b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today all of the tests are ran serially which on an M1 Max comes out to usually around 30+ seconds for me. This seems much too long to wait for what these tests actually do.. There was one decent deterrent in the way of getting these to actually run concurrently however, which is that today every test uses the exact same image, and it is unpacked every single time in the
bootstrapmethod. Every invocation after the first doesn't actually re-unpack the image, as the logic for "is this unpacked" is just a stat on the path, but what this means once we try and run the same on > 1 thread is they will all compete with each other on unpacking, and sometimes (almost always) trample over each other. This can result in guests trying to mount a faulty filesystem:[ 0.303404] EXT4-fs (vdb): VFS: Can't find ext4 filesystem
Ideally we have API to handle this, but for now I've invented a shoddy unpack coordinator type that just keys off of a path and halts any other threads if an unpack for that path is already in progress.
Timings (10 runs, image already pulled, max concurrency == 8):
New avg: 11s
Old Avg: 31s