Skip to content

CI: guard against silent test exclusions — fail when a tests/**/*Test.php is outside the unit/e2e allow-lists #476

Description

@s2x

Description

phpunit.xml defines the unit and e2e testsuites with an explicit allow-list of directories/files. Any future tests/**/*Test.php file that lands outside the listed paths is silently never executed by CI — no error, no warning. This exact failure mode produced #459: six real test files (182 tests, including the ResponseBuilder dispatch tests — the protocol routing core) sat unrun in the repo until someone noticed. #459 fixed the symptom (the six files now run); this issue prevents recurrence.

Per docs/workflow.md's "Prefer a gate over an entry" rule, this is a check, not a knowledge-base note.

Where

  • phpunit.xml:6-16 (the unit allow-list) and :18-20 (e2e)
  • CI gate: .github/workflows/ci.yml runs ./vendor/bin/phpunit --testsuite unit

Suggested fix

Add a small CI step (or composer script, e.g. composer test:suite-coverage) that diffs the set of discovered test files against the configured allow-list and fails if a tests/**/*Test.php is neither under a listed <directory>/<file> nor in the e2e suite. Sketch:

# files PHPUnit would discover
discovered=$(find tests -name '*Test.php' | sort)
# files the allow-lists cover (parse phpunit.xml, or just run --list-tests and diff)

Concretely, compare find tests -name '*Test.php' -not -path 'tests/E2E/*' against the union of the unit suite's <directory>/<file> entries; fail the build on any mismatch. Add it as a step in the lint job so it runs on every PR alongside the existing gates.

Why it matters

Without it, the next test file added outside the allow-list ships untested on green CI — the same invisible regression that #459 existed for an unknown time.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions