[Bug] `backgroundScope` coroutines can trigger `UncaughtExceptionsBeforeTest`

[viluon]

What is happening? What should have happened instead?

Currently, the backgroundScope in a runTest block is cancelled, but may leave coroutines running.

kotlinx.coroutines/kotlinx-coroutines-test/common/src/TestBuilders.kt

Line 369 in 8564f65

backgroundScope.cancel()

I believe its job should be .join()ed as well. Otherwise, subsequent tests can throw kotlinx.coroutines.test.UncaughtExceptionsBeforeTest.

Reproducer

See https://pl.kotl.in/okOqxIhaC

import kotlinx.coroutines.*
import kotlinx.coroutines.test.*

fun main() {
   runTest {
       backgroundScope.launch(Dispatchers.Default) {
           withContext(NonCancellable) {
               delay(1_000)
               error("hmm")
           }
       }

       delay(50)
   }

   Thread.sleep(1_500)

   runTest {
       println("I'm innocent")
   }
}

Code like this can occasionally crash with kotlinx.coroutines.test.UncaughtExceptionsBeforeTest, although I haven't been able to reproduce it in the playground.

Anything else? (optional)

I originally ran into this with a Ktor server launched in the backgroundScope when running tests in-process via the Mill build system, using isolated classloaders for the tests themselves but a shared classloader for ~clean dependencies, including kotlinx-coroutines-test. This led to a race between the Ktor server shutting down and Mill unloading the isolated classloader for the given test. When Mill was faster, the leftover coroutines in backgroundScope started running into NoClassDefFoundErrors, in turn triggering UncaughtExceptionsBeforeTest with the next runTest invocation, whatever that may have been.

[viluon]

The workaround seems to be joining the job manually with

backgroundScope.coroutineContext[Job]!!.cancelAndJoin()

before the end of the runTest block.

[AzazelSensei]

Opened #4726