Description
Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?
- Confirmed reproduction steps with SwiftPM CLI. The description text must include reproduction steps with either of command-line SwiftPM commands,
swift build,swift test,swift packageetc.
Description
I wrote a small test in both XCTest and Swift Testing
import XCTest
class SwiftTestingCrashResultXCTestCase: XCTestCase {
func testAlwaysPass() {
XCTAssert(true)
}
func testAlwaysFail() {
XCTAssert(false)
}
func testCrash() {
fatalError("This is a crash")
}
}import Testing
struct TestSuite {
@Test
func always_pass()
#expect(Bool(true))
}
@Test
func always_fail() {
#expect(Bool(false))
}
@Test
func crashing_test() {
assertionFailure("testing an assertionFailure")
}
}I put these in a small Swift Package as a test target, and ran using swift test --parallel --xunit-output xunit.xml
Expected behavior
The resulting xunit-swift-testing.xml should report always_pass() as passing, always_fail() as failing, and crashing_test() as failing as well. This would match the behavior of the equivalent XCTestCase.
Actual behavior
The xunit-swift-testing.xml contains only:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
While the xunit.xml file contains:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="TestResults" errors="0" tests="3" failures="2" time="0.156307999">
<testcase classname="SwiftTestingCrashResultTests.SwiftTestingCrashResultXCTestCase"
name="testCrash" time="0.052093333">
<failure message="failed"></failure>
</testcase>
<testcase classname="SwiftTestingCrashResultTests.SwiftTestingCrashResultXCTestCase"
name="testAlwaysPass" time="0.052104958">
</testcase>
<testcase classname="SwiftTestingCrashResultTests.SwiftTestingCrashResultXCTestCase"
name="testAlwaysFail" time="0.052109708">
<failure message="failed"></failure>
</testcase>
</testsuite>
</testsuites>
Steps to reproduce
Here's my package file:
// swift-tools-version: 6.0
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "SwiftTestingCrashResult",
dependencies: [
.package(url: "https://github.com/apple/swift-argument-parser.git", from: "1.2.0"),
],
targets: [
// Targets are the basic building blocks of a package, defining a module or a test suite.
// Targets can depend on other targets in this package and products from dependencies.
.executableTarget(
name: "SwiftTestingCrashResult",
dependencies: [
.product(name: "ArgumentParser", package: "swift-argument-parser"),
],
path: "Sources"
),
.testTarget(
name: "SwiftTestingCrashResultTests",
dependencies: ["SwiftTestingCrashResult"],
path: "Tests"
)
]
)The two tests are contained in a Tests/ folder in separate files, while the Sources/ folder contains the default starter code for a command-line tool:
// The Swift Programming Language
// https://docs.swift.org/swift-book
//
// Swift Argument Parser
// https://swiftpackageindex.com/apple/swift-argument-parser/documentation
import ArgumentParser
@main
struct SwiftTestingCrashResult: ParsableCommand {
mutating func run() throws {
print("Hello, world!")
}
}
Swift Package Manager version/commit hash
6.0
Swift & OS version (output of swift --version ; uname -a)
swift-driver version: 1.115.1 Apple Swift version 6.0.3 (swiftlang-6.0.3.1.10 clang-1600.0.30.1)
Target: arm64-apple-macosx15.0
Darwin Andys-MacBook-Pro.local 24.3.0 Darwin Kernel Version 24.3.0: Thu Jan 2 20:24:22 PST 2025; root:xnu-11215.81.4~3/RELEASE_ARM64_T6041 arm64