Skip to content

Test suite cleanup: isolate selftest from e2e, remove noise, unify coverage reporting #18

@codemonkeychris

Description

@codemonkeychris

Problem

We have three styles of tests — unit, selftest, and e2e — but their organization and quality have drifted:

  • Selftests are embedded inside the e2e suite. They aren't isolated in source, so running e2e requires filtering them out. Discovery and targeted runs are harder than they should be.
  • The e2e suite contains noise. A number of tests appear unused or duplicative, and it's unclear which ones are validating actual product functionality versus ones that were generated speculatively by AI assistants and landed without scrutiny.
  • No consistent code coverage reporting across the three test styles. We can't currently answer "what does our combined test coverage actually look like?" with a single number.

The net effect: tests are padding a vanity metric rather than giving us confidence that the product works.

Proposed work

  • Separate selftests from e2e in the source tree (distinct folders/projects) so each style runs independently without filtering.
  • Audit the e2e suite: for each test, identify the functionality under test and the failure mode it would catch. Remove or rewrite tests that don't map to real behavior.
  • Audit the unit and selftest suites the same way.
  • Establish a single coverage-reporting story that aggregates across unit + selftest + e2e, with clear thresholds per layer.
  • Document the distinction between the three test styles (when to write which) so new tests land in the right bucket.

Out of scope

Adding new test coverage for untested features — that's a follow-up once the existing suite is trustworthy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions