Fix NioAsyncWriter test on concurrency thread pool with single thread #3135
Conversation
|
@finagolfin : Could you verify this fix on the Android emulator? @Lukasa : I wanted to get some feedback first before writing tests for |
I can confirm this pull fixes the failing test on the Android emulator, 😃 once I updated the NIO package manifest to add this dependency. Ignore the failing trunk build: a new snapshot was just tagged which requires a three-hour build on my CI, so I manually canceled that trunk build. The 6.0/6.1 linux runs passed with the emulator set to a single core, meaning this pull fixed that, whereas the mac runs now show other tests failing but this test passes. I don't know what the other mac test failures are about, but I've been seeing a lot of mac flakiness in the last couple hours, so I'm hoping that's a mac CI issue that will go away. 😉 |
orobio
force-pushed
the
fix-NIOAsyncWriter-test-on-concurrency-thread-pool-with-single-thread
branch
from
3b535ec to
4957b3b
Compare
|
@finagolfin : That is good news. Thank you for verifying! I've added the dependency in Package.swift. @Lukasa : Would this solution be acceptable? Using |
orobio
force-pushed
the
fix-NIOAsyncWriter-test-on-concurrency-thread-pool-with-single-thread
branch
from
4957b3b to
66ec6ba
Compare
The testSuspendingBufferedYield_whenWriterFinished test causes the test application to hang in the Android emulator. This fix sets a timeout on waiting for the ConditionLock, so we don't wait indefinitely. Additionally, the timeout for waiting for both yields being suspended is increased to make it less likely that an incorrect state is reached.
Provides withNIOThreadPoolTaskExecutor, which runs a task executor based on a NIOThreadPool with a specified number of threads.
…ndroid emulator The testSuspendingBufferedYield_whenWriterFinished test requires at least two threads in the concurrency thread pool because it blocks one task, which waits for another task to set a condition. In environments where the global concurrency thread pool doesn’t have at least two threads available, the test will fail, as observed on the Android emulator running with a single virtual core (see discussion in apple#3044). Using a custom task executor guarantees that at least two threads are available for the test, regardless of the width of the global concurrency thread pool.
orobio
force-pushed
the
fix-NIOAsyncWriter-test-on-concurrency-thread-pool-with-single-thread
branch
from
66ec6ba to
7af7145
Compare
Co-authored-by: George Barnett <[email protected]>
Co-authored-by: George Barnett <[email protected]>
…d-pool-with-single-thread
…d-pool-with-single-thread
Head branch was pushed to by a user without write access
orobio
force-pushed
the
fix-NIOAsyncWriter-test-on-concurrency-thread-pool-with-single-thread
branch
from
c58b62e to
8afbc3f
Compare
| /// Enqueue a job. | ||
| /// | ||
| /// Called by the concurrency runtime. | ||
| /// | ||
| /// - Parameter job: The job to enqueue. | ||
| public func enqueue(_ job: UnownedJob) { | ||
| self.storage.withLock { storage in | ||
| if storage.isShutdown { | ||
| fatalError("A job is enqueued after manual executor shutdown") | ||
| } | ||
| storage.jobs.append(job) | ||
| } | ||
| } | ||
|
|
||
| /// Shutdown. | ||
| /// | ||
| /// Since the manual task executor is not running anything in the background, this is purely to catch | ||
| /// any issues due to incorrect usage of the executor. The shutdown verifies that the queue is empty | ||
| /// and makes sure that no new jobs can be enqueued. | ||
| @usableFromInline | ||
| func shutdown() { | ||
| self.storage.withLock { storage in | ||
| if !storage.jobs.isEmpty { | ||
| fatalError("Shutdown of manual executor with jobs in queue") | ||
| } | ||
| storage.isShutdown = true | ||
| } | ||
| } |
There was a problem hiding this comment.
I don't know if we really need this complicated shutdown handling. Since task executors can only be set on a scope we shouldn't have to do any manual clean up or reject jobs.
There was a problem hiding this comment.
It can catch some incorrect usage, like for example:
await withManualTaskExecutor { taskExecutor in
Task(executorPreference: taskExecutor) {
try await Task.sleep(for: .seconds(3))
}
taskExecutor.runUntilQueueIsEmpty() // Runs until the sleep starts, but the task is not finished yet
}It will definitely not catch all problems, but I was thinking there could be value in catching some.
…d-pool-with-single-thread
|
Looks like a few CI failures need cleaning up. |
Head branch was pushed to by a user without write access
Should be fixed now, except for this one, which doesn't seem related: |
|
@Lukasa : Would we need to do anything for the third issue? Or can we rerun the CI? |
…d-pool-with-single-thread
|
@orobio -- apologies for missing this one. I've just kicked the CI and enabled auto-merge. The last issue you linked above is a known issue which shouldn't block the merge. |
|
It looks like another issue popped up, but I'm not sure what to do with that one either. |
|
Looks like an infra problem, I've re-run that job. |
|
Something is up with that runner unrelated to your PR so I'm going to merge over the failure. |
Motivation:
The testSuspendingBufferedYield_whenWriterFinished test fails on the Android emulator. See also the discussion in #3044.
Modifications:
The test requires at least two threads in the concurrency thread pool because it blocks one task, which waits for another task to set a condition. This PR adds support for running a task executor based on a NIOThreadPool and uses it for the test. Using a custom task executor guarantees that at least two threads are available for the test.
Additionally, the test has been renamed to testWriterFinish_AndSuspendBufferedYield, which is more in line with the other test names.
Result:
The test will pass regardless of the width of the global concurrency thread pool.