feat: skip the Swift Testing tests a skip list names

AGENTS.md

@@ -4,7 +4,7 @@ A Swift package that test targets link to record which coverage counters each te
 
 ## Layout
 - `Sources/TestCoverageAttributionObserver` - Objective-C, Foundation only. XCTest is looked up at run time (class, protocol and selectors by name), so nothing links XCTest. It reads each instrumented image's `__llvm_prf_cnts` directly and never calls the LLVM profile runtime. Registers its XCTest observer from a constructor when the test bundle loads, and hooks `-[XCUIApplication launch]`/`-activate` once the bundle starts, to list the app processes UI tests launch. Its constructor also reads the skip list (`TEST_COVERAGE_ATTRIBUTION_SKIP_FILE`) and hooks `+[XCTestCase defaultTestSuite]` to remove the listed tests; it lives in this file because `-ObjC` loads only archive members that define a class.
-- `Sources/TestCoverageAttribution` - the Swift Testing trait (`.coverageAttribution`) that calls `test_coverage_attribution_scope_begin/end` around each test. It's kept but undocumented (`@_documentation(visibility: private)`, not in the README): test-scoped evidence and skipping are supported for XCTest and XCUITest only.
+- `Sources/TestCoverageAttribution` - the Swift Testing trait (`.coverageAttribution`) that calls `test_coverage_attribution_scope_begin/end` around each test, and cancels the tests `test_coverage_attribution_is_skipped` finds in the skip list. Cancelling needs `Test.cancel`, so it's compiled only with Swift 6.3 or later; before, listed tests run, since a test a trait doesn't run is reported as passed.
 - `Tests/TestCoverageAttributionTests` - runs the fixtures with attribution on and parses what the observer wrote: `Records.swift` mirrors the `records.bin` layout and `FunctionTable.swift` maps counters to function names as Tuist's reader does.
   - `Fixtures/Example` - with SwiftPM, xcodebuild on macOS and xcodebuild on an iOS simulator.
   - `Fixtures/Concurrency` - the overlapping, nested-trait and failure cases, in a package of their own since Swift Testing runs suites in parallel across the process.

Fixtures/UITests/AppTests/SkipListTests.swift

@@ -1,3 +1,4 @@
+import TestCoverageAttribution
 import Testing
 import XCTest
 
@@ -44,7 +45,18 @@ final class UnlistedTests: XCTestCase {
     func testRuns() {}
 }
 
-/// Listed too, but Swift Testing tests don't go through XCTest's suites, so it runs.
+/// The trait cancels the listed tests from Swift 6.3 on, which reports them as skipped; before,
+/// they run.
+@Suite(.coverageAttribution)
 struct SwiftTestingTests {
     @Test func runs() {}
+
+    @Test func skipped() {}
+
+    @Test(arguments: [1, 2]) func skippedWithArguments(value _: Int) {}
+}
+
+/// Listed too, but without the trait nothing reads the list, so it runs.
+struct UntraitedSwiftTestingTests {
+    @Test func runs() {}
 }

README.md

@@ -9,7 +9,7 @@ TestCoverageAttribution records which code each test executes. Xcode's code cove
 
 [Tuist](https://tuist.dev) uses it to collect per-test coverage evidence, which lets a run that skips tests reuse those tests' coverage from an earlier run, and to skip the tests a change can't affect.
 
-Both work for XCTest tests, UI tests included. Swift Testing tests run as usual: they get no per-test evidence and are never skipped one by one.
+Both work for XCTest tests, UI tests included. Swift Testing tests get them in suites that have the `.coverageAttribution` trait: evidence when a test doesn't run at the same time as another, and skipping with Swift 6.3 (Xcode 26.4) or later.
 
 ## Installation
 
@@ -43,7 +43,19 @@ The package records on Apple platforms only. Elsewhere, such as Linux, it builds
 
 **UI tests** need nothing else either. The app a UI test drives runs in a process of its own, which the package can't observe, but Xcode profiles that process separately when code coverage is on. The package records which test launched which app process, so a reader such as Tuist can attribute the app's coverage to the test. See [`launches.tsv`](#output).
 
-**Swift Testing** isn't supported. Swift Testing tests run as usual, and their coverage is in the run's report as always, but no test-scoped evidence is recorded for them and the skip list doesn't apply to them. Swift Testing has no observation center a library can join, it runs tests in parallel within a process by default, and a trait has no public way to skip a test.
+**Swift Testing** needs the `.coverageAttribution` trait on each suite, because Swift Testing has no observation center a library can join. The trait applies to the suites nested in the one that has it.
+
+```swift
+import TestCoverageAttribution
+import Testing
+
+@Suite(.coverageAttribution)
+struct DiscountTests {
+    @Test func appliesDiscount() { … }
+}
+```
+
+Swift Testing runs tests in parallel within a process by default, and the counters can't tell tests that run at the same time apart: their records are marked overlapped, and a reader should give them no evidence. A test gets evidence when no other test runs at the same time, as in a run with parallelization off; `.serialized` only orders a suite's own tests. Tests in suites without the trait run as usual and get none.
 
 ## When it records
 
@@ -68,7 +80,7 @@ Supported platforms are macOS and the iOS simulator. On a device, the process ca
 
 ## Skipping tests
 
-A runner that knows which tests a change can't affect, such as Tuist's coverage-based selective testing, can have the package remove them before XCTest runs them. Set `TEST_COVERAGE_ATTRIBUTION_SKIP_FILE` to the absolute path of a UTF-8 file that lists them, one per line. A relative path is read from the test process's working directory, such as the simulator app's, where the file isn't.
+A runner that knows which tests a change can't affect, such as Tuist's coverage-based selective testing, can have the package skip them before they run. Set `TEST_COVERAGE_ATTRIBUTION_SKIP_FILE` to the absolute path of a UTF-8 file that lists them, one per line. A relative path is read from the test process's working directory, such as the simulator app's, where the file isn't.
 
 ```text
 CheckoutTests/DiscountTests/testAppliesDiscount()
@@ -85,7 +97,7 @@ TEST_RUNNER_TEST_COVERAGE_ATTRIBUTION_SKIP_FILE="$PWD/skip.txt" xcodebuild test
 - **Forgiving.** Blank lines and tests that don't exist are ignored. An unreadable file skips nothing. Skipping doesn't depend on `TEST_COVERAGE_ATTRIBUTION_DIR`.
 - **Where it applies.** `xcodebuild` passes the variable to the test process with the `TEST_RUNNER_` prefix: the app that hosts unit tests, and the runner of UI tests. It works with `test` and `test-without-building`, with `-only-testing`, and with parallel testing, where Xcode lists the tests, without the removed ones, before it hands them out to its clones. `swift test` reads it from the environment.
 - **What a skipped test looks like.** It's removed from the suite XCTest builds for its class, so it doesn't run and isn't in the result bundle. A class whose tests are all removed doesn't run its class `setUp` and `tearDown`. A class with tests left runs them as usual, class `setUp` included.
-- **XCTest only.** Swift Testing tests in the same target run as usual, even when listed.
+- **Swift Testing.** In a suite with the `.coverageAttribution` trait, a listed test is cancelled before it starts, every argument of a parameterized test included, and reported as skipped, whether tests run in parallel or not. The line names it as the records do: the Swift module, the innermost suite, and the test's display name when it declares one (`Applies a discount`), its function otherwise (`appliesDiscount()`). A display name with a `/` can't be listed. This needs Swift 6.3 (Xcode 26.4) or later: before, a trait can skip a test only by not running it, which reports it as passed, so the listed tests run. Listed tests in suites without the trait run too.
 
 ## Output
 
@@ -119,7 +131,7 @@ The deltas are what turns counters into lines. In the image's coverage mapping (
 
 ## Limitations
 
-- **Swift Testing isn't attributed or skipped.** See [Usage](#usage).
+- **Swift Testing needs the trait.** Tests in suites without it aren't attributed or skipped, tests that run at the same time get no evidence, and skipping them needs Swift 6.3. See [Usage](#usage).
 - **Skipping some of a class's tests uses a private method.** XCTest has no public way to remove tests from a class's suite, so the package uses `XCTestSuite`'s private `-setTests:`, which keeps the suite and the class `setUp` it runs. If a future Xcode removes it, a class with some tests listed runs all of them; a class with all of its tests listed is still skipped. A class that overrides `defaultTestSuite` without calling `super` isn't filtered.
 - **UI tests record which app process each test launched, not the app's counters.** The app's coverage comes from the profile Xcode writes for each app process (`<uuid>-<pid>.profraw` beside the run's `Coverage.profdata`). The pid is read from `XCUIApplication`'s private `processID` property; if a future Xcode removes it, nothing is written. Processes the app starts itself, such as extensions, aren't listed.
 - **Images loaded late aren't observed.** An image loaded after the first test starts, such as a framework the tests `dlopen`, is missing from the records.

Sources/TestCoverageAttribution/CoverageAttributionTrait.swift

@@ -4,14 +4,11 @@
 import Testing
 
 /// Attributes the code each Swift Testing test executes to that test, by marking where each test
-/// starts and ends: Swift Testing has no observation center a library can join.
+/// starts and ends: Swift Testing has no observation center a library can join. It also skips the
+/// tests the skip list names, by cancelling them before they run.
 ///
-/// Kept, but hidden from the documentation and the README: test-scoped evidence and skipping are
-/// supported for XCTest and XCUITest only. The trait has to be added to every suite, Swift Testing
-/// runs tests in parallel within a process by default, which leaves overlapping tests without
-/// evidence, and a trait has no public way to skip a test. Projects that use it anyway get records
-/// as before.
-@_documentation(visibility: private)
+/// The trait has to be added to every suite, and Swift Testing runs tests in parallel within a
+/// process by default, which leaves overlapping tests without evidence.
 public struct CoverageAttributionTrait: TestTrait, SuiteTrait, TestScoping {
     /// Whether a scope is already open around the running test case: a test in a suite nested in
     /// another that has the trait gets it once from each, and two scopes would mark it overlapped.
@@ -37,6 +34,13 @@ public struct CoverageAttributionTrait: TestTrait, SuiteTrait, TestScoping {
             // The name the result bundle reports the test under, which is the test's identity
             // everywhere else: its display name when it declares one, its function otherwise.
             let name = test.displayName ?? components.last ?? test.name
+            #if compiler(>=6.3)
+                // `Test.cancel` reports the test as skipped. Before Swift 6.3 a trait can only
+                // skip a test by not running it, which reports it as passed, so it runs instead.
+                if test_coverage_attribution_is_skipped(module, suite, name) {
+                    try Test.cancel("Listed in TEST_COVERAGE_ATTRIBUTION_SKIP_FILE", sourceLocation: test.sourceLocation)
+                }
+            #endif
             test_coverage_attribution_scope_begin(module, suite, name)
             defer { test_coverage_attribution_scope_end(module, suite, name) }
             try await Self.$isInScope.withValue(true) {
@@ -50,7 +54,6 @@ public struct CoverageAttributionTrait: TestTrait, SuiteTrait, TestScoping {
 
 extension Trait where Self == CoverageAttributionTrait {
     /// See ``CoverageAttributionTrait``.
-    @_documentation(visibility: private)
     public static var coverageAttribution: Self {
         Self()
     }

Sources/TestCoverageAttributionObserver/TestCoverageAttributionObserver.m

@@ -26,9 +26,9 @@
 //   launches.tsv   one line per process a UI test launched, see record_launch()
 //
 // Skipping: $TEST_COVERAGE_ATTRIBUTION_SKIP_FILE (through xcodebuild,
-// TEST_RUNNER_TEST_COVERAGE_ATTRIBUTION_SKIP_FILE), an absolute path, names a file of XCTest tests
-// not to run, which are removed before XCTest runs them. See load_skip_list(). It works with or
-// without recording.
+// TEST_RUNNER_TEST_COVERAGE_ATTRIBUTION_SKIP_FILE), an absolute path, names a file of tests not to
+// run: XCTest tests are removed before XCTest runs them, and the trait cancels Swift Testing tests.
+// See load_skip_list(). It works with or without recording.
 #import <Foundation/Foundation.h>
 #import "TestCoverageAttributionObserver.h"
 #include <dlfcn.h>
@@ -406,8 +406,9 @@ static bool is_class_suite(NSString *name) {
 // The listed tests are removed from +[XCTestCase defaultTestSuite], the suite XCTest builds for
 // each class, both when it lists the tests (parallel runs hand the listed tests out to their
 // clones) and when it runs them. A removed test isn't in the result bundle, and a class whose
-// tests are all removed doesn't run its class setUp. Only XCTest is affected: Swift Testing tests
-// don't go through XCTest's suites.
+// tests are all removed doesn't run its class setUp. Swift Testing tests don't go through XCTest's
+// suites: the `.coverageAttribution` trait asks test_coverage_attribution_is_skipped() and cancels
+// the listed ones.
 static NSSet<NSString *> *skipped_tests;
 static IMP original_default_test_suite;
 
@@ -437,6 +438,13 @@ static void load_skip_list(void) {
     if (tests.count > 0) skipped_tests = tests;
 }
 
+// The list is read once, by the constructor, and never changes after: no lock is needed.
+bool test_coverage_attribution_is_skipped(const char *module, const char *suite, const char *name) {
+    if (!skipped_tests || !module || !suite || !name) return false;
+    NSString *identifier = skip_identifier(@(module), @(suite), @(name));
+    return identifier && [skipped_tests containsObject:identifier];
+}
+
 // A class's suite holds its tests. When some are listed, the suite is kept and its tests replaced
 // through XCTestSuite's private -setTests:, since the suite is what runs the class setUp and
 // tearDown; an XCTest without it runs every test. When all are listed, an empty suite takes its

Sources/TestCoverageAttributionObserver/include/TestCoverageAttributionObserver.h

@@ -1,6 +1,8 @@
 #ifndef TEST_COVERAGE_ATTRIBUTION_OBSERVER_H
 #define TEST_COVERAGE_ATTRIBUTION_OBSERVER_H
 
+#include <stdbool.h>
+
 #ifdef __cplusplus
 extern "C" {
 #endif
@@ -13,6 +15,10 @@ void test_coverage_attribution_scope_begin(const char *module, const char *suite
 /// Ends the scope `test_coverage_attribution_scope_begin` started and records it.
 void test_coverage_attribution_scope_end(const char *module, const char *suite, const char *name);
 
+/// Whether the skip list (`TEST_COVERAGE_ATTRIBUTION_SKIP_FILE`) names the test, matched the way
+/// XCTest tests are.
+bool test_coverage_attribution_is_skipped(const char *module, const char *suite, const char *name);
+
 #ifdef __cplusplus
 }
 #endif

Tests/TestCoverageAttributionTests/AttributionTests.swift

@@ -142,20 +142,26 @@ struct AttributionTests {
     }
 
     /// Runs one of the packages under `Fixtures`; with SwiftPM, only the tests `filter` matches.
-    /// `inspect` gets the output directory before it is removed.
+    /// `skipList` is the skip file's contents. `inspect` gets the output directory before it is
+    /// removed.
     /// `Concurrency` is a package of its own: Swift Testing runs suites in parallel across the
     /// process, so its tests would overlap the others.
-    private func runFixture(
+    func runFixture(
         _ runner: Runner,
         attributing: Bool,
         fixture name: String = "Example",
         filter: String? = nil,
+        skipList: String? = nil,
         inspect: (URL) throws -> Void = { _ in },
     ) throws -> [Record] {
         let scratch = FileManager.default.temporaryDirectory.appending(path: "TestCoverageAttribution-\(UUID().uuidString)")
         defer { try? FileManager.default.removeItem(at: scratch) }
         let output = scratch.appending(path: "output")
         try FileManager.default.createDirectory(at: output, withIntermediateDirectories: true)
+        let skipFile = scratch.appending(path: "skip.txt")
+        if let skipList {
+            try skipList.write(to: skipFile, atomically: true, encoding: .utf8)
+        }
 
         let fixture = Self.packageRoot.appending(path: "Fixtures/\(name)")
         let build = Self.packageRoot.appending(path: ".build/fixtures/\(name)")
@@ -164,12 +170,14 @@ struct AttributionTests {
         switch runner {
         case .swiftPM:
             environment["TEST_COVERAGE_ATTRIBUTION_DIR"] = attributing ? output.path() : nil
+            environment["TEST_COVERAGE_ATTRIBUTION_SKIP_FILE"] = skipList == nil ? nil : skipFile.path()
             arguments = [
                 "swift", "test", "--package-path", fixture.path(), "--scratch-path", build.appending(path: "swiftpm").path(),
                 "--enable-code-coverage",
             ] + (filter.map { ["--filter", $0] } ?? [])
         case .xcodebuild, .iOSSimulator:
             environment["TEST_RUNNER_TEST_COVERAGE_ATTRIBUTION_DIR"] = attributing ? output.path() : nil
+            environment["TEST_RUNNER_TEST_COVERAGE_ATTRIBUTION_SKIP_FILE"] = skipList == nil ? nil : skipFile.path()
             let destination = runner == .iOSSimulator ? "id=\(try Self.iPhoneSimulator(fixture: fixture))" : "platform=macOS"
             arguments = [
                 "xcodebuild", "test", "-scheme", "\(name)-Package", "-destination", destination,