-
-
Notifications
You must be signed in to change notification settings - Fork 610
Reimplement polling expectations to make them more reliable under heavy system loads #1199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
f7f6ff1 to
0aa5b7f
Compare
…vy system loads This trades wall-clock time accuracy for reliability. It's nowhere near as important for a polling expectation to finish within the specified duration as it is to actually poll as expected. Basically, under heavy system loads, the actual gap between polling attempts increases. Under sufficiently heavy loads, with a sufficiently low timeout, we might never have polled in the first place. Instead, this calculate how many times we should poll by dividing the timeout by the interval. We then poll exactly that many times, sleeping for however long the polling interval is. If polling interval is ever 0, then we set it to 1 nanosecond. WaitUntil is also re-implemented in both async and sync, but that's because the older infrastructure for doing polling was replaced. Starting in Swift 6.3, we will emit a warning when WaitUntil is used in Swift Testing. This change also removes the ability to detect a blocked runloop, so Nimble will no longer detect for that. In my opinion, this is a worthwhile tradeoff, as blocked runloop was a source of test flakiness.
Fix build errors on older xcodes/linux
0aa5b7f to
a265638
Compare
b1f97b6 to
afa847d
Compare
|
Any chance we can get a release published since this change was merged? |
|
👋 I tried to use the fix in my project and noticed a few test cases failing on sync In all cases |
|
@younata is there any way to get a release published with this change? This change fixes a ton of flakiness in our test suite of 15k tests. |
|
@tahirmt I still haven't gotten around to addressing @sewerynplazuk's issue. My apologies, I've had a lot going on recently. |
|
#1202 addresses the issues that @sewerynplazuk brought up. Once I merge that (tomorrow? It's 11:41 pm for me right now), I'll get version 14 out. |
This trades wall-clock time accuracy for reliability. It's nowhere near as important for a polling expectation to finish within the specified duration as it is to actually poll as expected.
Basically, under heavy system loads, the actual gap between polling attempts increases. Under sufficiently heavy loads, with a sufficiently low timeout, we might never have polled in the first place.
Instead, this calculate how many times we should poll by dividing the timeout by the interval. We then poll exactly that many times, sleeping for however long the polling interval is.
If polling interval is ever 0, then we set it to 1 nanosecond.
WaitUntil is also re-implemented in both async and sync, but that's because the older infrastructure for doing polling was replaced. Starting in Swift 6.3, we will emit a warning when WaitUntil is used in Swift Testing.
This change also removes the ability to detect a blocked runloop, so Nimble will no longer detect for that. In my opinion, this is a worthwhile tradeoff, as blocked runloop was a source of test flakiness.
This isn't changing any public-facing code, aside from adding deprecation warnings. It still counts as a major release.
Fixes #1095, though that's a happy consequence of this.