chore: fix make test-json hang under Swift 6.3.1 + restore CI test execution (#172)

* ci: switch test target filter to underscore form for Swift 6.3.1

Verified empirically: the prior dash form `Container-Compose-StaticTests` matches **zero tests** under Swift 6.3.1 (which normalizes module names dashes -> underscores in the test filter grammar). The 5 most recent CI runs on main / phase-4c branches all silently reported:

  warning: No matching test cases were run
  Executed 0 tests, with 0 failures
  Test run with 0 tests in 0 suites passed after 0.001 seconds
  conclusion: success

Both matrix targets (Static + Dynamic) were affected. Switching to the underscore form `Container_Compose_StaticTests` / `Container_Compose_DynamicTests` makes `swift test --filter` actually match the intended targets. PR 170 + CHAOS-1507 follow-up surfaced this.

* chore(makefile): switch test-json to working --filter pattern (drop --experimental-event-stream-output)

Under Swift 6.3.1 the prior `make test-json` hangs indefinitely:

* `--experimental-event-stream-output .build/test-events.jsonl` opens a socket-backed event stream that errors out without an active reader, deadlocking the whole `swift test` invocation. Tail of `.build/test-events.jsonl` shows the socket errors when this hits.
* `--skip Container-Compose-DynamicTests` is non-functional in 6.3.1 — the dash form matches no targets (same root cause as the CI workflow fix in the previous commit).

New target uses positive `--filter Container_Compose_StaticTests --no-parallel`:

* Underscore form actually selects the static target (verified locally — dash form returns 'No matching test cases were run').
* `--no-parallel` is explicit (not just defaulted) because multiple suites `dup2` global STDOUT to a Pipe to capture warnings (ResourceArgsTests, LifecycleArgsTests, GpusBlkioTests, SecurityArgsTests, NetworkArgsTests, ComposePort runtime-argv, ComposeDown orphan-volume recovery, ComposeUp block-image migration guard, ShutdownWatchdog) and race on the shared FD under `--parallel`.
* JSONL + `swift run test-report` pipeline is removed — plain swift-testing stdout is the canonical agent-readable surface again.

* docs(agents): update test invocation guidance for Swift 6.3.1

Reflect the fixes from the two preceding commits:

* `make test-json` now uses `swift test --filter Container_Compose_StaticTests --no-parallel` (underscore form because Swift 6.3.1 normalizes module-name dashes; `--no-parallel` because multiple static suites `dup2` STDOUT under parallel and race on the shared FD).
* JSONL + `swift run test-report` pipeline removed — plain swift-testing stdout is the canonical agent-readable surface again. Exit code is the plain `swift test` exit code.
* Expanded list of dup2-using suites (ResourceArgsTests, LifecycleArgsTests, GpusBlkioTests, SecurityArgsTests, NetworkArgsTests, ComposePort runtime-argv, ComposeDown orphan-volume recovery, ComposeUp block-image migration guard, ShutdownWatchdog) is documented so contributors know not to re-introduce `--parallel` to the local test target.

Cross-references PR #170 + CHAOS-1507 follow-up where this was empirically verified.

* fix(tests): reset warn-once dedup in SecurityFeatureIntegrationTests

`SecurityArgsTests` runs alphabetically before `SecurityFeatureIntegrationTests` and exercises `service.privileged`, `service.security_opt`, and `service.group_add` via `SecurityArgs.build()`, consuming the warnings in the process-global `warnUnsupportedRuntimeFieldOnce` dedup set. When the static suite runs serially via `make test-json` / `swift test --no-parallel`, `SecurityFeatureIntegrationTests` then asserts that `captureStdout(...).contains("Note: ...")` succeeds — but the dedup gate suppresses the second emission, so captured stdout is empty and three tests fail.

Mirror the pattern from `LifecycleArgsTests.init`: reset the warn-once set in `init()`. The suite is already `.serialized` so the reset runs deterministically before each test.

Surfaced by `--filter Container_Compose_StaticTests` (the underscore form Swift 6.3.1 actually matches). The legacy dashed filter on main matched zero tests and silently passed.

* fix(runtime): honor absolute paths + guard empty argv in ProductionRunner.resolveBinary

`resolveBinary(forArgv:)` previously routed everything-not-`"container-compose"` to the `container` binary unconditionally. The `LineBufferTests` PLAN.md §4 regression guard (`endToEndChunkedStdoutProducesOneLineAtATime`) drives `ProductionRunner` against `/bin/sh -c "..."` and was silently spawning `container -c "..."` instead, which exits 64 with `Error: unknown option '-c'` on any host where apple/container is installed.

Two changes, both no-ops for production callsites that always pass an unqualified program name:

1. argv[0] starting with `/` now short-circuits to `URL(fileURLWithPath: first)` and bypasses the cached resolver. Production never fires this branch; it exists so test code can drive a real `/bin/sh` subprocess through `ProductionRunner` without being silently re-routed.

2. Empty argv now throws `RuntimeError.backendFailure` via `guard let` instead of silently falling through to `containerBin` (which would launch the apple/container CLI with no arguments and print its help banner — a confusing diagnostic for what is really a caller bug).

Surfaced together with the SecurityFeatureIntegrationTests failures once `--filter Container_Compose_StaticTests` actually executed the suite under Swift 6.3.1.

* chore(ci): fail loudly when test filter matches < 1500 tests

CI's `swift test --filter Container-Compose-StaticTests` was a silent no-op: Swift 6.3.1 normalizes the dashed target name to underscores in the filter grammar, so the dashed form matched zero tests, swift test exited 0, and the job reported green. Earlier commits on this branch correct the filter to `Container_Compose_StaticTests`.

Add a min-tests assertion as defense-in-depth so a future filter typo, target rename, or Swift Testing grammar shift cannot silently regress the suite to a zero-match again:

- `.github/workflows/tests.yml`: matrix entries gain a `min_tests` field (1500 for Static, 0 for Dynamic — CI runners do not ship apple/container so Dynamic legitimately reports `0 tests`). The run step pipes through `tee`, parses the Swift Testing summary line for the executed count, and fails with `::error::` when count is below the floor.

- `Makefile`: `test-json` gets the same guard inline. Honors the upstream swift-test exit code; only overrides on the filter-matched-too-few condition.

- `AGENTS.md`: documents the guard alongside the existing `--parallel` caveat and the underscore-filter rationale.

Author: chrisgeo

.github/workflows/tests.yml

@@ -53,11 +53,23 @@ jobs:
       matrix:
         include:
           - suite: Static
-            target: Container-Compose-StaticTests
+            target: Container_Compose_StaticTests
             extra_args: "--parallel"
+            # Silent-skip guard (CHAOS-1507 follow-up). Swift 6.3.1 normalizes
+            # target-name dashes to underscores in the filter grammar; a typo or
+            # regression to the dashed form `Container-Compose-StaticTests`
+            # silently matches zero tests and exits 0. Fail loudly if fewer
+            # than `min_tests` ran. Floor chosen well below the current 1806
+            # so additions/removals don't trigger noise.
+            min_tests: 1500
           - suite: Dynamic
-            target: Container-Compose-DynamicTests
+            target: Container_Compose_DynamicTests
             extra_args: ""
+            # CI runners do not ship apple/container, so every dynamic test
+            # self-skips via `.enabled(if: RuntimeAvailability.isAvailable())`.
+            # Swift Testing reports the run as `0 tests`. No floor is
+            # enforceable here; we still build the target to catch compile drift.
+            min_tests: 0
 
     steps:
     - name: Checkout code
@@ -88,7 +100,22 @@ jobs:
       run: swift build --build-tests
 
     - name: Run ${{ matrix.suite }} tests
-      run: swift test --filter "${{ matrix.target }}" ${{ matrix.extra_args }}
+      shell: bash
+      run: |
+        set -o pipefail
+        log_file="test-output-${{ matrix.suite }}.log"
+        swift test --filter "${{ matrix.target }}" ${{ matrix.extra_args }} 2>&1 | tee "$log_file"
+        test_exit=${PIPESTATUS[0]}
+        # Swift Testing prints e.g. `Test run with 1806 tests in 151 suites passed`.
+        # XCTest prints e.g. `Executed N tests`. Capture either form.
+        count=$(grep -oE 'Test run with [0-9]+ tests|Executed [0-9]+ tests' "$log_file" | grep -oE '[0-9]+' | tail -1 || true)
+        count=${count:-0}
+        echo "::notice::Suite ${{ matrix.suite }} executed $count tests (min ${{ matrix.min_tests }})"
+        if [ "$count" -lt "${{ matrix.min_tests }}" ]; then
+          echo "::error::Filter '${{ matrix.target }}' executed $count tests (expected >= ${{ matrix.min_tests }}). Did the Swift Testing target-name normalization change, or did the filter typo into a zero-match form?"
+          exit 2
+        fi
+        exit $test_exit
 
     - name: Upload test artifacts on failure
       if: failure()

AGENTS.md

@@ -393,16 +393,28 @@ schema changes that ripple into `up`/`down`/`build`.
 Prefer `make test-json` over raw `swift test` when an agent (or any
 automation) is parsing results. From the [Makefile](./Makefile):
 
-- Routes the static suite through swift-testing's event stream and emits
-  structured JSON via `swift run test-report` — agent-parseable, no
-  `tail -N` truncation games.
-- Skips `Container-Compose-DynamicTests` automatically (equivalent to
-  `--filter Container-Compose-StaticTests` but with the right tooling).
-- Forbids `--parallel`. Several static suites (`LifecycleArgsTests`,
-  `ResourceArgsTests`, `GpusBlkioTests`) `dup2` global `STDOUT_FILENO`
-  to a `Pipe` to capture printed warnings; under `--parallel` they race
-  on the shared FD and `readDataToEndOfFile()` hangs indefinitely.
-- Exit codes: `0` = all passed, `1` = any failed, `2` = no events emitted.
+- Runs the static suite via `swift test --filter Container_Compose_StaticTests`
+  (underscore form — Swift 6.3.1 normalizes the dashed target name and the
+  legacy `Container-Compose-StaticTests` filter matches **zero tests**).
+  See PR #170 + the CHAOS-1507 follow-up.
+- The previous JSONL + `swift run test-report` pipeline was removed because
+  `--experimental-event-stream-output` now hangs indefinitely under Swift
+  6.3.1 when no socket reader is attached. Plain swift-testing stdout is the
+  canonical agent-readable surface again.
+- Forbids `--parallel`. Multiple suites `dup2` global `STDOUT_FILENO` to a
+  `Pipe` to capture printed warnings: `LifecycleArgsTests`, `ResourceArgsTests`,
+  `GpusBlkioTests`, `SecurityArgsTests`, `NetworkArgsTests`, the ComposePort
+  runtime-argv tests, ComposeDown orphan-volume recovery, the ComposeUp
+  block-image migration guard, and `ShutdownWatchdog`. Under `--parallel` they
+  race on the shared FD and `readDataToEndOfFile()` hangs indefinitely — do
+  not re-introduce `--parallel` to this target.
+- Exit code is the plain `swift test` exit code (0 = pass, non-zero = fail).
+- **Silent-skip guard.** Both `make test-json` and the CI workflow assert that
+  the static suite executes **>= 1500 tests** (current count: ~1806). If the
+  filter typo'd back to a zero-match form (e.g. the dashed `Container-Compose-StaticTests`)
+  the guard prints an `ERROR: filter executed only N tests` message and exits
+  2 — defending against the same footgun that caused CI to be a silent no-op
+  between the Swift 6.3.1 upgrade and the CHAOS-1507 follow-up.
 
 Previously, `VolumeMountIntegrationTests` could leak real Docker Hub pulls when its environment wraps missed `RunnerEnvironment.$current`. Fixed by wrapping `RecordingRunner()` in every test. All test `projectName` values are now prefixed `cc-test-` (e.g. `cc-test-vol-up-<uuid>`) to make test-originated container names unambiguous. When sweeping this convention to other static-suite test files, use the same `cc-test-` prefix. Leaving this note as a defensive reminder.
 
@@ -442,8 +454,9 @@ the broad `--no-parallel` workaround from CHAOS-1314.
 - **Don't break the topo sort.** `Service.topoSortConfiguredServices` is on
   the hot path of `up`. Cycle detection there must keep throwing.
 - **Commits.** Small, focused commits. Rebuild with `make build` and run
-  `make test-json` (or full `swift test` if your change touches dynamics)
-  before pushing.
+  `make test-json` (which is `swift test --filter Container_Compose_StaticTests`
+  under Swift 6.3.1; see PR #170 / CHAOS-1507) or full `swift test` if your
+  change touches dynamics before pushing.
 - **Background agents for long-running tasks.** Always dispatch test runs
   (prefer `make test-json` for structured, agent-parseable output; raw
   `swift test` only when you need the full run including
@@ -503,9 +516,11 @@ make build           # release config, copies binary to .build/release/
 make install         # symlinks into /usr/local/bin (sudo may be required)
 
 # Run tests
-make test-json                                     # static suite, structured JSON (preferred for agents)
+make test-json                                     # static suite only (Swift 6.3.1 underscore filter; preferred for agents)
 swift test                                         # full run including Container-Compose-DynamicTests
 swift test --filter LifecycleArgsTests             # single suite (replace with target)
+# Equivalent to `make test-json` if you need to invoke directly:
+swift test --filter Container_Compose_StaticTests  # underscore form — dashes match 0 tests under Swift 6.3.1 (see PR #170)
 
 # Use locally
 .build/release/container-compose up -f path/to/docker-compose.yml

Makefile

@@ -72,21 +72,38 @@ build-tests:
 test:
 	swift test
 
-# Run the static suite under swift-testing's event stream and emit a structured
-# report. Useful for agent-driven test reading or CI parsing. Defaults to
-# fast-feedback path: skipping the dynamic Apple-container target (use
-# `make test` for the full run including dynamics).
+# Run the static test suite only — fast feedback path for agents and CI-equivalent
+# local checks. Dynamic tests are gated by RuntimeAvailability so this filter is
+# the canonical way to skip them on hosts without apple/container.
 #
-# NOTE: --parallel intentionally NOT used. The static suite contains tests
-# (ResourceArgsTests, LifecycleArgsTests, GpusBlkioTests) that `dup2` global
-# STDOUT_FILENO to a Pipe to capture printed warnings; under --parallel they
-# race on the shared FD and `readDataToEndOfFile()` hangs indefinitely.
+# CHAOS-1507 follow-up ("make test-json hang"): under Swift 6.3.1 the prior
+# implementation used `--experimental-event-stream-output` to feed a JSONL file
+# into `swift run test-report`. That socket-backed event stream now hangs
+# indefinitely when no reader attaches in time, and `--skip Container-Compose-DynamicTests`
+# silently matches nothing because Swift 6.3.1 normalizes target dashes to
+# underscores in the test filter grammar. Both have been removed; we use the
+# underscore form `--filter Container_Compose_StaticTests` instead.
 #
-# Exit code mirrors the report: 0 = all passed, 1 = any failed, 2 = no events.
+# NOTE: --parallel intentionally NOT used. Multiple suites `dup2` global
+# STDOUT_FILENO to a Pipe to capture printed warnings (ResourceArgsTests,
+# LifecycleArgsTests, GpusBlkioTests, SecurityArgsTests, NetworkArgsTests,
+# ComposePort runtime-argv tests, ComposeDown orphan-volume recovery,
+# ComposeUp block-image migration guard, ShutdownWatchdog). Under --parallel
+# they race on the shared FD and `readDataToEndOfFile()` hangs indefinitely.
 test-json:
-	rm -f .build/test-events.jsonl
-	-swift test --skip Container-Compose-DynamicTests --experimental-event-stream-output .build/test-events.jsonl
-	swift run test-report .build/test-events.jsonl --format json
+	@set -o pipefail; \
+	mkdir -p .build; \
+	log_file=.build/test-json.log; \
+	swift test --filter Container_Compose_StaticTests --no-parallel 2>&1 | tee "$$log_file"; \
+	test_exit=$$?; \
+	count=$$(grep -oE 'Test run with [0-9]+ tests|Executed [0-9]+ tests' "$$log_file" | grep -oE '[0-9]+' | tail -1 || true); \
+	count=$${count:-0}; \
+	echo "test-json: executed $$count tests"; \
+	if [ "$$count" -lt 1500 ]; then \
+		echo "ERROR: filter executed only $$count tests (expected >= 1500). Did the Swift Testing target-name normalization change?"; \
+		exit 2; \
+	fi; \
+	exit $$test_exit
 
 # Regenerate coverage.json from the inline JSON in coverage.html.
 coverage:
@@ -108,7 +125,7 @@ help:
 	@echo "  debug             Debug build of $(binary_name)"
 	@echo "  build-tests       Compile tests without running (CI-equivalent)"
 	@echo "  test              Run all tests (dynamic ones self-skip without Apple container)"
-	@echo "  test-json         Run static tests + emit structured JSON report (skips dynamic; for agents/CI)"
+	@echo "  test-json         Run static suite only (Swift 6.3.1 underscore filter; CI-equivalent agent-friendly target)"
 	@echo "  coverage          Regenerate coverage.json from coverage.html"
 	@echo "  clean             Remove .build/"
 	@echo "  install           Build + install to \$$(bindir) (default $(bindir))"

Sources/Container-Compose/Runtime/RunCommandRunner.swift

@@ -203,8 +203,29 @@ public struct ProductionRunner: RunCommandRunner {
 
     /// Pick the right cached binary for an argv whose first element is the
     /// program name (`"container"` or `"container-compose"`).
+    ///
+    /// Absolute paths (e.g. `"/bin/sh"`) are honored as-is and bypass the cache.
+    /// Production callsites always pass an unqualified program name, so this
+    /// branch is a no-op in production; it exists so test code can drive
+    /// `ProductionRunner` against a real `/bin/sh` subprocess without being
+    /// silently re-routed to the apple/container CLI.
+    ///
+    /// Empty argv is a programmer error: every `RunRequest` must carry at
+    /// least one element. Fail loudly via `RuntimeError.backendFailure`
+    /// rather than silently routing an empty-args invocation to the
+    /// `container` binary (which would launch the CLI with no arguments and
+    /// print its help banner — a confusing diagnostic for what is really a
+    /// caller bug).
     fileprivate static func resolveBinary(forArgv argv: [String]) throws -> URL {
-        let cached: Result<URL, RuntimeError> = (argv.first == "container-compose") ? selfBin : containerBin
+        guard let first = argv.first else {
+            throw RuntimeError.backendFailure(
+                message: "ProductionRunner.resolveBinary: argv is empty"
+            )
+        }
+        if first.hasPrefix("/") {
+            return URL(fileURLWithPath: first)
+        }
+        let cached: Result<URL, RuntimeError> = (first == "container-compose") ? selfBin : containerBin
         return try cached.get()
     }
 

Tests/Container-Compose-StaticTests/SecurityFeatureIntegrationTests.swift

@@ -60,6 +60,17 @@ import Yams
 @Suite("Security feature integration tests — YAML → SecurityArgs.build() → argv", .serialized)
 struct SecurityFeatureIntegrationTests {
 
+    /// Reset the process-wide warn-once dedup set before each test so the
+    /// `captureStdout(...).contains("Note: ...")` assertions don't flake based
+    /// on which sibling suite ran first. Peer suite `SecurityArgsTests` also
+    /// exercises `service.privileged`, `service.security_opt`, and
+    /// `service.group_add` via `SecurityArgs.build()`, which would otherwise
+    /// consume the one-shot warning under serial execution. Mirrors the
+    /// pattern established in `LifecycleArgsTests.init`.
+    init() {
+        resetUnsupportedRuntimeFieldWarningsForTesting()
+    }
+
     // MARK: - Helpers
 
     /// Decode a `DockerCompose` from YAML and return the named service.