On Windows `swift test` does not exit until all subprocesses of the test function have exited

[ahoppen]

When an XCTest function launches a subprocess, swift test will not exit on Windows until that subprocess has terminated. For example, the test below will keep swift test running for 15s even though the xctest bundle finishes after 1s.

This is particularly problematic because the subprocesses that swift test is waiting for are now dangling processes without a parent. As far as I can tell, they thus don’t have anyone reading their stdout and stderr and can block if their pipes fill up, putting the swift test invocation into an infinite hang. We have seen SourceKit-LSP tests appear to run for days in CI because of this.

import XCTest
import Foundation

class MyTests: XCTestCase {
    func testSubprocessKeepsRunningInBackground() async throws {
        let subprocess = Process()
        // Use `ping` as a dummy process that takes a while to finish
        subprocess.executableURL = URL(fileURLWithPath: #"C:\Windows\System32\ping.exe"#)
        subprocess.arguments = ["1.1.1.1", "-n", "15"]
        try subprocess.run()
        try await Task.sleep(for: .seconds(1))
    }
}

[ahoppen]

Looking at this again, on macOS swift test also waits for all subprocesses to terminate. It just prints the test results (how many tests passed etc) when all the test functions have returned and before all subprocesses have finished, while on Windows the test results are only printed when all subprocesses have terminated.

Reproducer on macOS

import Foundation
import XCTest

class MyTests: XCTestCase {
    func testSubprocessKeepsRunningInBackground() async throws {
        Task {
            let subprocess = Process()
            subprocess.executableURL = URL(fileURLWithPath: #"/bin/sleep"#)
            subprocess.arguments = ["15"]
            try subprocess.run()
            subprocess.waitUntilExit()
        }
        try await Task.sleep(for: .seconds(1))
    }
}

[jakepetroules]

Note that waitUntilExit() is unsafe to call in an asynchronous context.