Skip to content

TIA: the Xdebug recorder traces vendor/ and then discards it, where the pcov path restricts up front #1865

Description

@studioycm

Summary

TIA's recorder starts Xdebug coverage with no path filter, traces every executed line including all of vendor/, and then discards everything out of scope in PHP. The stored graph holds zero vendor files, so all of that tracing is thrown away.

The pcov path in the same file does the opposite: it restricts collection to the in-scope list before materialising it.

On a real suite, excluding vendor/ from instrumentation cut the recording pass from 1,633s to 657s — the Xdebug tax dropped from 4.61× to 1.86× relative to a plain run. The recorded graph was unchanged.

The asymmetry

src/Plugins/Tia/Recorder.php:

// pcov — restricted BEFORE collect (:136-149)
\pcov\stop();
$scope = $this->sourceScope();
$filesToCollectCoverageFor = [];
foreach (\pcov\waiting() as $file) {
    if (is_string($file) && $scope->contains($file)) {
        $filesToCollectCoverageFor[] = $file;
    }
}
$data = \pcov\collect(\pcov\inclusive, $filesToCollectCoverageFor);
// xdebug — unfiltered, then discarded in PHP (:121, :152-160)
\xdebug_start_code_coverage();          // no filter, no arguments
...
$data = \xdebug_get_code_coverage();
\xdebug_stop_code_coverage(true);
$scope = $this->sourceScope();
foreach (array_keys($data) as $file) {
    if (! $scope->contains($file)) {     // discard vendor here, after tracing it

xdebug_set_filter does not appear anywhere in src/.

Measured (pest 5.1.0, PHPUnit 13.3.0, PHP 8.4.23, Xdebug 3.4.0alpha2-dev, macOS)

Suite: 2,032 tests, 354s without coverage.

run wall tax
--tia recording, unfiltered Xdebug 1,633s 4.61×
--tia recording, vendor/ excluded 657s 1.86×

The filter applied, in the project's Pest bootstrap:

xdebug_set_filter(XDEBUG_FILTER_CODE_COVERAGE, XDEBUG_PATH_EXCLUDE, [__DIR__.'/../vendor/']);

Correctness was checked before speed, since a filter that narrows the graph would be worse than the tax:

  • Graph shape unchanged — 945 files, 171 test files with edges, every top-level directory present with identical counts.
  • Per-test-file edge sets identical for 167 of 168 files; the one difference was a transient that self-corrected on the next run and involved two project files, which the filter cannot explain.
  • A filtered run of one edit went from 546s to 247s — from 1.54× slower than a plain full suite to 1.43× faster.

Suggested fix

Call xdebug_set_filter(XDEBUG_FILTER_CODE_COVERAGE, XDEBUG_PATH_EXCLUDE, [...]) — or the XDEBUG_PATH_INCLUDE equivalent derived from sourceScope() — before xdebug_start_code_coverage(), so Xdebug is asked not to instrument what the recorder is about to throw away.

An exclude list is the safer form: an include list built from a scope guess can silently narrow the graph, whereas excluding vendor/ can only remove files sourceScope() already rejects.

Note

Excluding vendor/ is what was measured. Excluding it and everything else out of scope may do better; that was not tested. The 4.61× and 1.86× figures are from one machine and one suite, so treat the ratio as indicative rather than a benchmark.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions