Skip to content

Collect deprecations triggered by autoloading while loading/building the test suite #6165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 4, 2025

Conversation

soyuka
Copy link
Contributor

@soyuka soyuka commented Mar 28, 2025

When one triggers a deprecation in a class directly like this:

<?php
trigger_error('Depreciated interface', E_USER_DEPRECATED);

interface Foo {}

This technique is notably used in many Symfony components, for example:

https://github.com/symfony/symfony/blob/82cf90aefb7e9117ebd981fec83d7546e310e52b/src/Symfony/Component/PropertyInfo/Type.php#L12-L15

The deprecation will be triggered when autoloading the class, which happens in:

$testSuite = $this->buildTestSuite($configuration);

At that point, the ErrorHandler is not registered and we loose the deprecation.

This pull request gives an idea on how it may be fixed, indeed the ErrorHandler needs a TestCase therefore I'm collecting these deprecations and triggering them once we call ErrorHandler::enable. I'm open to suggestions on how to fix this differently. Obviously I'll add tests if/once a solution is found that can be merged.

Thanks!

@@ -178,8 +178,16 @@ public function run(array $argv): int

EventFacade::instance()->seal();

$depreciations = [];

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this variable should be $deprecations.

@@ -422,4 +424,16 @@ private function stackTrace(): string

return $buffer;
}

public function collectGlobalDepreciations(array $deprecations): void

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this method should be collectGlobalDeprecations.

@sebastianbergmann
Copy link
Owner

sebastianbergmann commented Mar 28, 2025

I think this idea is worthwhile to pursue. Please fix the existing tests and add new tests to cover the changed/added functionality.

I do not consider this to be a bug fix, so this pull request should target the main branch instead of 11.5.

@sebastianbergmann sebastianbergmann added type/enhancement A new idea that should be implemented feature/test-runner CLI test runner labels Mar 28, 2025
@soyuka soyuka changed the base branch from 11.5 to main March 28, 2025 15:13
@soyuka soyuka force-pushed the global-deprec branch 4 times, most recently from d02f1ab to 10515dd Compare March 28, 2025 23:35
@soyuka
Copy link
Contributor Author

soyuka commented Mar 28, 2025

#5844 link I updated the code, interestingly this conflicts with the resolution of #5844 (https://github.com/sebastianbergmann/phpunit/commit/8e8776083c71277fa90403b183530c956b110b60). To mitigate this I need to use something like:

while (true) {
$previousHandler = set_error_handler(static fn () => false);
restore_error_handler();
if ($previousHandler === null) {
break;
}
$activeErrorHandlers[] = $previousHandler;
restore_error_handler();
}

The idea would be to restore only the handler that we set to avoid restoring another handler registered by third party (bootstrap or test suite configuration).

The handler that's created in the bootstrap.php of 5844 fails once restored as it's a private method... I also think it makes the code slightly more complicated then what I would like (see failure at https://github.com/sebastianbergmann/phpunit/actions/runs/14144107940/job/39629361311?pr=6165)

@soyuka soyuka force-pushed the global-deprec branch 4 times, most recently from 3a94e60 to af0dc23 Compare March 29, 2025 19:39
@@ -0,0 +1,12 @@
<?php declare(strict_types=1);

spl_autoload_register(function ($class) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reflects what happens when loading classes usually, if we do the same as in #5844 we'll trigger the deprecation during the loadBootstrapScript.

@soyuka soyuka force-pushed the global-deprec branch 4 times, most recently from 987f371 to 452fedf Compare April 2, 2025 08:59
@TimWolla
Copy link

Did you test this with OPcache? I came across this PR on my phone and php/php-src#17422 sounds relevant.

@soyuka
Copy link
Contributor Author

soyuka commented Apr 18, 2025

I don't use opcache

@sebastianbergmann sebastianbergmann force-pushed the main branch 4 times, most recently from 1bb9097 to 5af6639 Compare April 20, 2025 07:09
@soyuka
Copy link
Contributor Author

soyuka commented Apr 26, 2025

@sebastianbergmann tests added and code cleaned, let me know if I should do something else. Thanks!

@soyuka
Copy link
Contributor Author

soyuka commented Jun 3, 2025

I've added a new commit as I had an issue with the baseline trace, the CI failures looks unrelated though.

@sebastianbergmann
Copy link
Owner

The tests for OTR logging use information from the Git checkout and apparently Git checkouts for pull requests are "different". I do not get these failures locally.

However, locally I get this (after rebasing your branch against main):

❯ ./phpunit
PHPUnit 12.2-g38f62c3d4b by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.7
Configuration: /Users/sb/Work/OpenSource/phpunit/phpunit.xml

.............................................................   61 / 4440 (  1%)
.............................................................  122 / 4440 (  2%)
.............................................................  183 / 4440 (  4%)
.............................................................  244 / 4440 (  5%)
.............................................................  305 / 4440 (  6%)
.............................................................  366 / 4440 (  8%)
.............................................................  427 / 4440 (  9%)
.............................................................  488 / 4440 ( 10%)
.............................................................  549 / 4440 ( 12%)
.............................................................  610 / 4440 ( 13%)
.............................................................  671 / 4440 ( 15%)
.............................................................  732 / 4440 ( 16%)
.............................................................  793 / 4440 ( 17%)
.............................................................  854 / 4440 ( 19%)
.............................................................  915 / 4440 ( 20%)
.............................................................  976 / 4440 ( 21%)
............................................................. 1037 / 4440 ( 23%)
............................................................. 1098 / 4440 ( 24%)
............................................................. 1159 / 4440 ( 26%)
............................................................. 1220 / 4440 ( 27%)
............................................................. 1281 / 4440 ( 28%)
............................................................. 1342 / 4440 ( 30%)
............................................................. 1403 / 4440 ( 31%)
............................................................. 1464 / 4440 ( 32%)
............................................................. 1525 / 4440 ( 34%)
............................................................. 1586 / 4440 ( 35%)
............................................................. 1647 / 4440 ( 37%)
............................................................. 1708 / 4440 ( 38%)
............................................................. 1769 / 4440 ( 39%)
............................................................. 1830 / 4440 ( 41%)
............................................................. 1891 / 4440 ( 42%)
............................................................. 1952 / 4440 ( 43%)
............................................................. 2013 / 4440 ( 45%)
............................................................. 2074 / 4440 ( 46%)
............................................................. 2135 / 4440 ( 48%)
............................................................. 2196 / 4440 ( 49%)
............................................................. 2257 / 4440 ( 50%)
............................................................. 2318 / 4440 ( 52%)
............................................................. 2379 / 4440 ( 53%)
............................................................. 2440 / 4440 ( 54%)
............................................................. 2501 / 4440 ( 56%)
............................................................. 2562 / 4440 ( 57%)
............................................................. 2623 / 4440 ( 59%)
............................................................. 2684 / 4440 ( 60%)
............................................................. 2745 / 4440 ( 61%)
............................................................. 2806 / 4440 ( 63%)
............................................................. 2867 / 4440 ( 64%)
.........................................................DDDD 2928 / 4440 ( 65%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 2989 / 4440 ( 67%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3050 / 4440 ( 68%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3111 / 4440 ( 70%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3172 / 4440 ( 71%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3233 / 4440 ( 72%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3294 / 4440 ( 74%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3355 / 4440 ( 75%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3416 / 4440 ( 76%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3477 / 4440 ( 78%)
DDDDDDDDDDDDDDDD.....DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3538 / 4440 ( 79%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3599 / 4440 ( 81%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3660 / 4440 ( 82%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3721 / 4440 ( 83%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD 3782 / 4440 ( 85%)
DDDDDDDDDDDDDDDDDDDDDDDDDDDDD........F....................... 3843 / 4440 ( 86%)
............................................................. 3904 / 4440 ( 87%)
............................................................. 3965 / 4440 ( 89%)
.....S.....................................................S. 4026 / 4440 ( 90%)
............................................................. 4087 / 4440 ( 92%)
......S...................................................... 4148 / 4440 ( 93%)
............................................................. 4209 / 4440 ( 94%)
............................................................. 4270 / 4440 ( 96%)
.....................S.S..................................... 4331 / 4440 ( 97%)
..................S.......S.................................. 4392 / 4440 ( 98%)
I....................S..........................              4440 / 4440 (100%)

Time: 00:54.039, Memory: 56.00 MB

There was 1 failure:

1) /Users/sb/Work/OpenSource/phpunit/tests/end-to-end/baseline/use-baseline-ignore-deprecation.phpt
Failed asserting that string matches format description.
--- Expected
+++ Actual
@@ @@
 PHPUnit 12.2-g38f62c3d4b by Sebastian Bergmann and contributors.

-Runtime: %s
-Configuration: %s
-
-.                                                               1 / 1 (100%)
-
-Time: %s, Memory: %s
-
-OK (1 tests, 1 assertions)
-
-1 issues were ignored by baseline.
+Could not read XML from file "/Users/sb/Work/OpenSource/phpunit/tests/end-to-end/baseline/../_files/baseline/use-baseline-ignore-deprecation/phpunit.xml"

/Users/sb/Work/OpenSource/phpunit/tests/end-to-end/baseline/use-baseline-ignore-deprecation.phpt:15
/Users/sb/Work/OpenSource/phpunit/src/Framework/TestSuite.php:374
/Users/sb/Work/OpenSource/phpunit/src/Framework/TestSuite.php:374
/Users/sb/Work/OpenSource/phpunit/src/TextUI/TestRunner.php:64
/Users/sb/Work/OpenSource/phpunit/src/TextUI/Application.php:221

FAILURES!
Tests: 4440, Assertions: 11766, Failures: 1, Deprecations: 1, Skipped: 8, Incomplete: 1.

@sebastianbergmann
Copy link
Owner

Please rebase after #6225. Thank you!

@soyuka
Copy link
Contributor Author

soyuka commented Jun 4, 2025

rebased, this one works well on our project now. Side note, this patch reports global deprecations only when using spl_autoload. When using require (see my comment here the deprecation gets triggered at loadBootstrapScript(at that point the error handler isn't registered IIRC).

In my opinion we should document that global deprecations are reported only when using the autoload mechanism, as that triggers the error only when we use the class, which is what maintainers want to report.

@sebastianbergmann
Copy link
Owner

I am sorry, but running PHPUnit's own test suite with the changes proposed here triggers a lot of deprecations: https://github.com/sebastianbergmann/phpunit/actions/runs/15436395576/job/43443720453?pr=6165

@soyuka
Copy link
Contributor Author

soyuka commented Jun 4, 2025

nice catch, indeed I forgot to clean up the ErrorHandler instance in the unit test.

@sebastianbergmann sebastianbergmann changed the title collect autoloading deprecations before running testCase Collect deprecations triggered by autoloading while loading/building the test suite Jun 4, 2025
@sebastianbergmann sebastianbergmann added this to the PHPUnit 12.2 milestone Jun 4, 2025
@sebastianbergmann sebastianbergmann merged commit 086064b into sebastianbergmann:main Jun 4, 2025
22 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/test-runner CLI test runner type/enhancement A new idea that should be implemented
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants