TIA: a suite absent from the graph can never be recorded while a graph exists — replay only refreshes tests the graph already knows

[dllsystem]

Summary

Once a TIA graph exists, a test suite (or any set of test files) the graph does not know about can never be recorded into it — no matter how many times it runs with --tia. Replay mode is entered solely because a graph exists, and inside replay the recorder is only activated for tests the graph already knows. The unknown tests execute (reported as uncached), but neither their edges nor their results are persisted — so every subsequent run pays the full uncached cost again, forever.

This bites hardest in the common Laravel setup where two PHPUnit configs share one project: the default phpunit.xml (Unit/Feature) and a second config for browser tests (phpunit.browser.xml, different testsuite directory). Whichever suite records first wins the graph; the other one is permanently locked out of it.

Reproduction

Any project with Pest v5, a coverage driver (pcov), and git. No browser plugin needed — the point is a test directory outside the recorded graph.

# 1. Two configs, two test directories
#    phpunit.xml          -> <testsuite name="Unit"><directory>tests/Unit</directory></testsuite>
#    phpunit.browser.xml  -> same file, but <testsuite name="Browser"><directory>tests/Browser</directory></testsuite>
mkdir -p tests/Browser
cp phpunit.xml phpunit.browser.xml   # then edit the testsuite as above

# 2. One trivial test in each directory
#    tests/Unit/ExampleTest.php      -> test('unit', fn () => expect(true)->toBeTrue());
#    tests/Browser/ExampleTest.php   -> test('browser', fn () => expect(true)->toBeTrue());

# 3. Commit everything so the working tree is clean (important: $changed must be [])
git add -A && git commit -m repro

# 4. Record the graph with the default config
vendor/bin/pest --tia
#    -> "Experimental TIA mode enabled / fresh graph."
#    -> .pest/tia/graph.json (or ~/.pest default) now has edges for tests/Unit only

# 5. Run the second suite — any number of times
vendor/bin/pest -c phpunit.browser.xml --tia
#    -> banner says "Experimental TIA mode enabled." (replay — no "fresh graph")
#    -> the Browser test RUNS, reported as "1 uncached"

# 6. Check the graph
grep -c 'tests/Browser' .pest/tia/graph.json
#    -> 0. Repeat step 5 forever: still 0. The suite is never cached.

Observed at scale (real project)

Laravel app, Pest v5.1.1, pcov, 8.3k Feature tests + 446 browser tests (Pest browser plugin), both configs sharing the default TIA directory:

• Feature leg records first (445 s, fresh graph), writes graph.json.
• Browser leg then runs --tia: enters replay, executes all 446 tests as uncached in 950 s, and adds zero entries to the graph — no edges, no results (baselines[branch].results gained nothing for those files).
• Ran it twice to confirm (950 s and 922 s): grep -c 'tests/Browser' graph.json → 0 both times.
• Net effect: the browser suite pays full price on every single run while believing it is cached.

Mechanism (v5.1.1 sources)

1. src/Plugins/Tia.php:905-910 — the record/replay gate is graph existence, nothing else:

if ($graph instanceof Graph) {
    return $this->enterReplayMode($graph, $projectRoot, $arguments);
}

return $this->enterRecordMode($arguments);

2. Inside enterReplayMode(), the affected set is derived from the graph's own edges (Tia.php:1054):

$affectedFromChanges = $changed === [] ? [] : $graph->testFilesOnDisk($graph->affected($changed));

A test file the graph has never seen cannot be produced by $graph->affected() — so it is never part of $affected, regardless of working-tree state. With a clean tree it is even simpler: $changed === [] → $affected === [].

3. Recording during replay is gated on that same set (Tia.php:1076):

$canRefreshReplayEdges = $affected !== [] && $coverageAvailable;

With $affected === [], RECORDING_GLOBAL is never set, workers never activate the recorder, and snapshotTestResults() has nothing marked to persist for the unknown files. The unknown tests still execute (they are not replayable), but the run is write-only-for-known-tests: edges and results for the unknown suite are dropped on the floor.

So the system has exactly two stable states — "no graph: record everything" and "graph: refresh only what I already know" — and there is no path from the second state to "learn something new", except deleting the graph (--fresh), which throws away the first suite's cache to record the second's.

Expected

Either of these would fix it:

• Treat test files unknown to the graph as affected. During replay, scan the selected testsuite for files absent from graph.json and include them in $affected (recorder active for them). Unknown tests already run anyway — the only change is that their edges/results get persisted instead of discarded.
• Or give each resolved configuration its own graph. This is the same input-set gap as TIA: the fingerprint ignores the config actually in use (-c / --configuration) #1858 (the fingerprint ignores -c/--configuration): two configs with disjoint testsuites currently share one graph and one fingerprint, so they can neither coexist nor invalidate each other.

Workaround

Per-suite graph directories, switched by an env var:

// tests/Pest.php
pest()->tia()->locally()->directory(env('PEST_TIA_DIR', '.pest/tia'));

# browser suite runs
PEST_TIA_DIR=.pest/tia-browser vendor/bin/pest -c phpunit.browser.xml --tia

Each suite then records fresh once and replays afterwards. Works, but every consumer script must remember to pass the env var — forgetting it silently re-creates the bug.

Environment

• pestphp/pest v5.1.1 (also inspected 5.x up to 2026-08-18 — 9b36f44b, 2e589182 don't touch this logic)
• PHP 8.4, pcov (xdebug also present), Linux, git 2.43
• Related: TIA: the fingerprint ignores the config actually in use (-c / --configuration) #1858 (fingerprint ignores the config actually in use — same root cause family: the graph/fingerprint don't account for which configuration produced them)