Skip to content

Speed up allocation counting tests #3197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Conversation

glbrntt
Copy link
Contributor

@glbrntt glbrntt commented Apr 11, 2025

Motivation:

The allocation counting tests take around 23 minutes in CI, which is quite a while! The tests themselves don't take nearly so long, however each test is run 11 times but between each time the framework waits for allocations to become rebalanced (i.e. everything we allocated should be freed). The framework will wait up to 5 seconds between each iteration for this. About 15 or so tests take 55 seconds, meaning they're just waiting for the allocs to quiece. It turns out that dispatch doesn't always free blocks from async work items immediately. This is relevant because a number of tests create their own MTELGs, and so when they are shutdown we see allocations from the shutdown flow lingering for more than 5 seconds.

Not only does this slow the tests down, if the allocations don't quiece, the framework ignores the result, so we lose some coverage too.

Modifiications:

  • Add a pre-warmed singleton MTELG which tests can use, this ensures it has plenty of space in its queues so that intermediate allocs from running tests aren't seen as leaks
  • Cancel tasks which are scheduled to run after the test to avoid 'leaks'.
  • Avoid using swift run and just execute the test binary, this shaves off a few seconds for each test.
  • Print out how long each test run takes to make debugging easier
  • Fix a bug where the framework prints out logs from all tests when checking the results of each test

Result:

  • Faster alloc tests
  • 'Integration tests' CI job is ~10 minutes faster

Motivation:

The allocation counting tests take around 18 minutes in CI, which is
quite a while! The tests themselves don't take nearly so long, however
each test is run 11 times but between each time the framework waits for
allocations to become rebalanced (i.e. everything we allocated should
be freed). The framework will wait up to 5 seconds between each
iteration for this. About 15 or so tests take 55 seconds, meaning
they're just waiting for the allocs to quiece. It turns out that
dispatch doesn't always free blocks from `async` work items immediately.
This is relevant because a number of tests create their own MTELGs, and
so when they are shutdown we see allocations from the shutdown flow
lingering for more than 5 seconds.

Not only does this slow the tests down, if the allocations don't quiece,
the framework ignores the result, so we lose some coverage too.

Modifiications:

- Add a pre-warmed singleton MTELG which tests can use, this ensures it
  has plenty of space in its queues so that intermediate allocs from
  running tests aren't seen as leaks
- Cancel tasks which are scheduled to run after the test to avoid
  'leaks'.
- Avoid using `swift run` and just execute the test binary, this shaves
  off a few seconds for each test.
- Print out how long each test run takes to make debugging easier

Result:

- Faster alloc tests
@glbrntt glbrntt added the semver/none No version bump required. label Apr 11, 2025
@glbrntt glbrntt marked this pull request as ready for review April 11, 2025 19:39
@glbrntt glbrntt enabled auto-merge (squash) April 14, 2025 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver/none No version bump required.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant