Collect deprecations triggered by autoloading while loading/building the test suite #6165

soyuka · 2025-03-28T11:21:31Z

When one triggers a deprecation in a class directly like this:

<?php
trigger_error('Depreciated interface', E_USER_DEPRECATED);

interface Foo {}

This technique is notably used in many Symfony components, for example:

https://github.com/symfony/symfony/blob/82cf90aefb7e9117ebd981fec83d7546e310e52b/src/Symfony/Component/PropertyInfo/Type.php#L12-L15

The deprecation will be triggered when autoloading the class, which happens in:

phpunit/src/TextUI/Application.php

Line 181 in 4b6a4ee

$testSuite = $this->buildTestSuite($configuration);

At that point, the ErrorHandler is not registered and we loose the deprecation.

This pull request gives an idea on how it may be fixed, indeed the ErrorHandler needs a TestCase therefore I'm collecting these deprecations and triggering them once we call ErrorHandler::enable. I'm open to suggestions on how to fix this differently. Obviously I'll add tests if/once a solution is found that can be merged.

Thanks!

sebastianbergmann · 2025-03-28T12:16:28Z

src/TextUI/Application.php

@@ -178,8 +178,16 @@ public function run(array $argv): int

            EventFacade::instance()->seal();

+            $depreciations = [];


The name of this variable should be $deprecations.

sebastianbergmann · 2025-03-28T12:17:20Z

src/Runner/ErrorHandler.php

@@ -422,4 +424,16 @@ private function stackTrace(): string

        return $buffer;
    }
+
+    public function collectGlobalDepreciations(array $deprecations): void


The name of this method should be collectGlobalDeprecations.

sebastianbergmann · 2025-03-28T12:18:22Z

I think this idea is worthwhile to pursue. Please fix the existing tests and add new tests to cover the changed/added functionality.

I do not consider this to be a bug fix, so this pull request should target the main branch instead of 11.5.

soyuka · 2025-03-28T23:48:46Z

#5844 link

I updated the code, interestingly this conflicts with the resolution of #5844 (https://github.com/sebastianbergmann/phpunit/commit/8e8776083c71277fa90403b183530c956b110b60). To mitigate this I need to use something like:

phpunit/src/Framework/TestCase.php

Lines 1598 to 1610 in dd3bb4b

    
           while (true) { 
        
               $previousHandler = set_error_handler(static fn () => false); 
        
               restore_error_handler(); 
        
               if ($previousHandler === null) { 
        
                   break; 
        
               } 
        
               $activeErrorHandlers[] = $previousHandler; 
        
               restore_error_handler(); 
        
           }

The idea would be to restore only the handler that we set to avoid restoring another handler registered by third party (bootstrap or test suite configuration).

The handler that's created in the bootstrap.php of 5844 fails once restored as it's a private method... I also think it makes the code slightly more complicated then what I would like (see failure at https://github.com/sebastianbergmann/phpunit/actions/runs/14144107940/job/39629361311?pr=6165)

soyuka · 2025-03-29T20:18:06Z

tests/end-to-end/deprecation-trigger/_files/deprecation-trigger-class/vendor/autoload.php

@@ -0,0 +1,12 @@
+<?php declare(strict_types=1);
+
+spl_autoload_register(function ($class) {


This reflects what happens when loading classes usually, if we do the same as in #5844 we'll trigger the deprecation during the loadBootstrapScript.

TimWolla · 2025-04-11T14:06:09Z

Did you test this with OPcache? I came across this PR on my phone and php/php-src#17422 sounds relevant.

soyuka · 2025-04-18T07:23:07Z

I don't use opcache

soyuka · 2025-04-26T19:22:53Z

@sebastianbergmann tests added and code cleaned, let me know if I should do something else. Thanks!

soyuka · 2025-06-03T07:45:24Z

I've added a new commit as I had an issue with the baseline trace, the CI failures looks unrelated though.

sebastianbergmann · 2025-06-03T15:27:14Z

The tests for OTR logging use information from the Git checkout and apparently Git checkouts for pull requests are "different". I do not get these failures locally.

However, locally I get this (after rebasing your branch against main):

❯ ./phpunit
PHPUnit 12.2-g38f62c3d4b by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.7
Configuration: /Users/sb/Work/OpenSource/phpunit/phpunit.xml

.............................................................   61 / 4440 (  1%)
.............................................................  122 / 4440 (  2%)
.............................................................  183 / 4440 (  4%)
.............................................................  244 / 4440 (  5%)
.............................................................  305 / 4440 (  6%)
.............................................................  366 / 4440 (  8%)
.............................................................  427 / 4440 (  9%)
.............................................................  488 / 4440 ( 10%)
.............................................................  549 / 4440 ( 12%)
.............................................................  610 / 4440 ( 13%)
.............................................................  671 / 4440 ( 15%)
.............................................................  732 / 4440 ( 16%)
.............................................................  793 / 4440 ( 17%)
.............................................................  854 / 4440 ( 19%)
.............................................................  915 / 4440 ( 20%)
.............................................................  976 / 4440 ( 21%)
............................................................. 1037 / 4440 ( 23%)
............................................................. 1098 / 4440 ( 24%)
............................................................. 1159 / 4440 ( 26%)
............................................................. 1220 / 4440 ( 27%)
............................................................. 1281 / 4440 ( 28%)
............................................................. 1342 / 4440 ( 30%)
............................................................. 1403 / 4440 ( 31%)
............................................................. 1464 / 4440 ( 32%)
............................................................. 1525 / 4440 ( 34%)
............................................................. 1586 / 4440 ( 35%)
............................................................. 1647 / 4440 ( 37%)
............................................................. 1708 / 4440 ( 38%)
............................................................. 1769 / 4440 ( 39%)
............................................................. 1830 / 4440 ( 41%)
............................................................. 1891 / 4440 ( 42%)
............................................................. 1952 / 4440 ( 43%)
............................................................. 2013 / 4440 ( 45%)
............................................................. 2074 / 4440 ( 46%)
............................................................. 2135 / 4440 ( 48%)
............................................................. 2196 / 4440 ( 49%)
............................................................. 2257 / 4440 ( 50%)
............................................................. 2318 / 4440 ( 52%)
............................................................. 2379 / 4440 ( 53%)
............................................................. 2440 / 4440 ( 54%)
............................................................. 2501 / 4440 ( 56%)
............................................................. 2562 / 4440 ( 57%)
............................................................. 2623 / 4440 ( 59%)
............................................................. 2684 / 4440 ( 60%)
............................................................. 2745 / 4440 ( 61%)
............................................................. 2806 / 4440 ( 63%)
............................................................. 2867 / 4440 ( 64%)
.........................................................DDDD 2928 / 4440 ( 65%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 2989 / 4440 ( 67%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3050 / 4440 ( 68%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3111 / 4440 ( 70%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3172 / 4440 ( 71%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3233 / 4440 ( 72%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3294 / 4440 ( 74%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3355 / 4440 ( 75%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3416 / 4440 ( 76%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3477 / 4440 ( 78%)
DDDDDDDDDDDDDDDD.....DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3538 / 4440 ( 79%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3599 / 4440 ( 81%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3660 / 4440 ( 82%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3721 / 4440 ( 83%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3782 / 4440 ( 85%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDD........F....................... 3843 / 4440 ( 86%)
............................................................. 3904 / 4440 ( 87%)
............................................................. 3965 / 4440 ( 89%)
.....S.....................................................S. 4026 / 4440 ( 90%)
............................................................. 4087 / 4440 ( 92%)
......S...................................................... 4148 / 4440 ( 93%)
............................................................. 4209 / 4440 ( 94%)
............................................................. 4270 / 4440 ( 96%)
.....................S.S..................................... 4331 / 4440 ( 97%)
..................S.......S.................................. 4392 / 4440 ( 98%)
I....................S..........................              4440 / 4440 (100%)

Time: 00:54.039, Memory: 56.00 MB

There was 1 failure:

1) /Users/sb/Work/OpenSource/phpunit/tests/end-to-end/baseline/use-baseline-ignore-deprecation.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 PHPUnit 12.2-g38f62c3d4b by Sebastian Bergmann and contributors.

-Runtime: %s
-Configuration: %s
-
-.                                                               1 / 1 (100%)
-
-Time: %s, Memory: %s
-
-OK (1 tests, 1 assertions)
-
-1 issues were ignored by baseline.
+Could not read XML from file "/Users/sb/Work/OpenSource/phpunit/tests/end-to-end/baseline/../_files/baseline/use-baseline-ignore-deprecation/phpunit.xml"

/Users/sb/Work/OpenSource/phpunit/tests/end-to-end/baseline/use-baseline-ignore-deprecation.phpt:15
/Users/sb/Work/OpenSource/phpunit/src/Framework/TestSuite.php:374
/Users/sb/Work/OpenSource/phpunit/src/Framework/TestSuite.php:374
/Users/sb/Work/OpenSource/phpunit/src/TextUI/TestRunner.php:64
/Users/sb/Work/OpenSource/phpunit/src/TextUI/Application.php:221

FAILURES!
Tests: 4440, Assertions: 11766, Failures: 1, Deprecations: 1, Skipped: 8, Incomplete: 1.

sebastianbergmann · 2025-06-04T07:26:10Z

Please rebase after #6225. Thank you!

soyuka · 2025-06-04T07:43:12Z

rebased, this one works well on our project now. Side note, this patch reports global deprecations only when using spl_autoload. When using require (see my comment here the deprecation gets triggered at loadBootstrapScript(at that point the error handler isn't registered IIRC).

In my opinion we should document that global deprecations are reported only when using the autoload mechanism, as that triggers the error only when we use the class, which is what maintainers want to report.

sebastianbergmann · 2025-06-04T07:55:49Z

I am sorry, but running PHPUnit's own test suite with the changes proposed here triggers a lot of deprecations: https://github.com/sebastianbergmann/phpunit/actions/runs/15436395576/job/43443720453?pr=6165

soyuka · 2025-06-04T09:37:40Z

nice catch, indeed I forgot to clean up the ErrorHandler instance in the unit test.

sebastianbergmann reviewed Mar 28, 2025

View reviewed changes

sebastianbergmann added type/enhancement A new idea that should be implemented feature/test-runner CLI test runner labels Mar 28, 2025

soyuka changed the base branch from 11.5 to main March 28, 2025 15:13

soyuka force-pushed the global-deprec branch 4 times, most recently from d02f1ab to 10515dd Compare March 28, 2025 23:35

soyuka force-pushed the global-deprec branch 4 times, most recently from 3a94e60 to af0dc23 Compare March 29, 2025 19:39

soyuka commented Mar 29, 2025

View reviewed changes

soyuka requested a review from sebastianbergmann March 29, 2025 20:18

soyuka force-pushed the global-deprec branch 4 times, most recently from 987f371 to 452fedf Compare April 2, 2025 08:59

nicolas-grekas mentioned this pull request Apr 13, 2025

Add PHPUnit 11 recipe, treat PHPUnitBridge only as a PHPUnit extension symfony/recipes#1401

Merged

sebastianbergmann force-pushed the main branch 4 times, most recently from 1bb9097 to 5af6639 Compare April 20, 2025 07:09

soyuka force-pushed the global-deprec branch from 452fedf to 47c90ed Compare June 3, 2025 07:19

soyuka force-pushed the global-deprec branch from 47c90ed to b2aa417 Compare June 3, 2025 16:34

soyuka added 2 commits June 4, 2025 09:34

collect autoloading deprecations before running testCase

fdd7967

guess deprecation frame before checking for ignored baseline

233632d

soyuka force-pushed the global-deprec branch from b2aa417 to 233632d Compare June 4, 2025 07:35

clean up error handler instance

086064b

sebastianbergmann changed the title ~~collect autoloading deprecations before running testCase~~ Collect deprecations triggered by autoloading while loading/building the test suite Jun 4, 2025

sebastianbergmann added this to the PHPUnit 12.2 milestone Jun 4, 2025

sebastianbergmann merged commit 086064b into sebastianbergmann:main Jun 4, 2025
22 of 23 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Collect deprecations triggered by autoloading while loading/building the test suite #6165

Collect deprecations triggered by autoloading while loading/building the test suite #6165

Uh oh!

soyuka commented Mar 28, 2025

Uh oh!

sebastianbergmann Mar 28, 2025

Uh oh!

sebastianbergmann Mar 28, 2025

Uh oh!

sebastianbergmann commented Mar 28, 2025 •

edited

Loading

Uh oh!

soyuka commented Mar 28, 2025 •

edited

Loading

Uh oh!

soyuka Mar 29, 2025

Uh oh!

TimWolla commented Apr 11, 2025

Uh oh!

soyuka commented Apr 18, 2025

Uh oh!

soyuka commented Apr 26, 2025

Uh oh!

soyuka commented Jun 3, 2025

Uh oh!

sebastianbergmann commented Jun 3, 2025

Uh oh!

sebastianbergmann commented Jun 4, 2025

Uh oh!

soyuka commented Jun 4, 2025 •

edited

Loading

Uh oh!

sebastianbergmann commented Jun 4, 2025

Uh oh!

soyuka commented Jun 4, 2025

Uh oh!

Uh oh!

Uh oh!

		@@ -178,8 +178,16 @@ public function run(array $argv): int

		EventFacade::instance()->seal();

		$depreciations = [];

		@@ -0,0 +1,12 @@
		<?php declare(strict_types=1);

		spl_autoload_register(function ($class) {

Uh oh!

Collect deprecations triggered by autoloading while loading/building the test suite #6165

Collect deprecations triggered by autoloading while loading/building the test suite #6165

Uh oh!

Conversation

soyuka commented Mar 28, 2025

Uh oh!

sebastianbergmann Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

sebastianbergmann Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

sebastianbergmann commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soyuka commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soyuka Mar 29, 2025

Choose a reason for hiding this comment

Uh oh!

TimWolla commented Apr 11, 2025

Uh oh!

soyuka commented Apr 18, 2025

Uh oh!

soyuka commented Apr 26, 2025

Uh oh!

soyuka commented Jun 3, 2025

Uh oh!

sebastianbergmann commented Jun 3, 2025

Uh oh!

sebastianbergmann commented Jun 4, 2025

Uh oh!

soyuka commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sebastianbergmann commented Jun 4, 2025

Uh oh!

soyuka commented Jun 4, 2025

Uh oh!

Uh oh!

Uh oh!

sebastianbergmann commented Mar 28, 2025 •

edited

Loading

soyuka commented Mar 28, 2025 •

edited

Loading

soyuka commented Jun 4, 2025 •

edited

Loading