Tests fail on PHP 7.3

[legoktm]

Using PHP 7.3 on Debian unstable, I see the following test failures:

There were 18 failures:

1) Symfony\Polyfill\Tests\Apcu\ApcuTest::testApcu
Failed asserting that false is true.

/home/km/projects/symfony-polyfill/tests/Apcu/ApcuTest.php:27

2) Symfony\Polyfill\Tests\Apcu\ApcuTest::testArrayCompatibility
Failed asserting that false matches expected 'value1'.

/home/km/projects/symfony-polyfill/tests/Apcu/ApcuTest.php:52

3) Symfony\Polyfill\Tests\Apcu\ApcuTest::testAPCUIterator
Failed asserting that false is true.

/home/km/projects/symfony-polyfill/tests/Apcu/ApcuTest.php:71

4) Symfony\Polyfill\Tests\Intl\Idn\IdnTest::testDecodeInvalid2003 with data set #0 ('xn--zcaccffbljjkknnoorrssuuxx...g9g.de', 'xn--zcaccffbljjkknnoorrssuuxx...g9g.de')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'xn--zcaccffbljjkknnoorrssuuxxd5e0a0a3ae9c6a4a9bzdzdxdudwdxd2d2d8d0dse7d6dwe9dxeueweye4eyewe9e5ewkkewc9ftfpfplwexfwf4infvf2f6f6f7f8fpg8fmgngrgrgvgzgygxg3gyg1g3g5gykqg9g.de'
+'äöüßáàăâåãąāæćĉčċçďđéèĕêěëėęēğĝġģĥħíìĭîïĩįīıĵķĺľļłńňñņŋóòŏôőõøōœĸŕřŗśŝšşťţŧúùŭûůűũųūŵýŷÿźžżðþ.de'

/home/km/projects/symfony-polyfill/tests/Intl/Idn/IdnTest.php:83

5) Symfony\Polyfill\Tests\Intl\Idn\IdnTest::testDecodeInvalid2003 with data set #1 ('xn--zcaccffbljjkknnoorrssuuxx...vda.de', 'xn--zcaccffbljjkknnoorrssuuxx...g.þ.de')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'xn--zcaccffbljjkknnoorrssuuxxd5e0a0a3ae9c8c1b0dxdvdvdxdvd3d0d6dyd8d5d4due7dveseuewe2eweue7e3esk9dxc7frf9e7kuevfuf1ilftf5f4f4f5f6fng6f8f9fpgpgtgxgwgvg1g2gzg1g3gvkog7g.þ.de'
+'äöüßáàăâåãąāæćĉčċçďđéèĕêěëėęēğĝġģĥħíìĭîïĩįīıĵķĺľļłńňñņŋóòŏôőõøōœĸŕřŗśŝšşťţŧúùŭûůűũųūŵýŷÿźžżð.þ.de'

/home/km/projects/symfony-polyfill/tests/Intl/Idn/IdnTest.php:83

6) Symfony\Polyfill\Tests\Intl\Idn\IdnTest::testUppercase2003 with data set #0 ('рф.RU', 'xn--p1ai.RU', 'рф.RU')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'xn--p1ai.RU'
+'xn--p1ai.ru'

/home/km/projects/symfony-polyfill/tests/Intl/Idn/IdnTest.php:93

7) Symfony\Polyfill\Tests\Intl\Idn\IdnTest::testUppercase2003 with data set #1 ('GUANGDONG.广东', 'GUANGDONG.xn--xhq521b', 'GUANGDONG.广东')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'GUANGDONG.xn--xhq521b'
+'guangdong.xn--xhq521b'

/home/km/projects/symfony-polyfill/tests/Intl/Idn/IdnTest.php:93

8) Symfony\Polyfill\Tests\Intl\Idn\IdnTest::testUppercase2003 with data set #2 ('renanGonçalves.COM', 'xn--renangonalves-pgb.COM', 'renangonçalves.COM')
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'xn--renangonalves-pgb.COM'
+'xn--renangonalves-pgb.com'

/home/km/projects/symfony-polyfill/tests/Intl/Idn/IdnTest.php:93

9) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testConstants
Failed asserting that Array &0 (
    'FORM_C' => 4
    'FORM_D' => 2
    'FORM_KC' => 5
    'FORM_KD' => 3
    'NFC' => 4
    'NFD' => 2
    'NFKC' => 5
    'NFKD' => 3
    'NONE' => 1
) is identical to Array &0 (
    'FORM_C' => 16
    'FORM_D' => 4
    'FORM_KC' => 32
    'FORM_KC_CF' => 48
    'FORM_KD' => 8
    'NFC' => 16
    'NFD' => 4
    'NFKC' => 32
    'NFKC_CF' => 48
    'NFKD' => 8
    'NONE' => 2
).

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:37

10) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testIsNormalized
Failed asserting that false is true.

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:51

11) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testNormalize
Failed asserting that false is identical to 'déjà훈쇼™'.

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:68

12) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testNormalizeConformance
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'Ḋ'
+'Ḋ'

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:109

13) Symfony\Polyfill\Tests\Apcu\ApcuTest::testApcu
Polyfills enabled, Failed asserting that false is true.

/home/km/projects/symfony-polyfill/tests/Apcu/ApcuTest.php:27

14) Symfony\Polyfill\Tests\Apcu\ApcuTest::testArrayCompatibility
Polyfills enabled, Failed asserting that false matches expected 'value1'.

/home/km/projects/symfony-polyfill/tests/Apcu/ApcuTest.php:52

15) Symfony\Polyfill\Tests\Apcu\ApcuTest::testAPCUIterator
Polyfills enabled, Failed asserting that false is true.

/home/km/projects/symfony-polyfill/tests/Apcu/ApcuTest.php:71

16) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testConstants
Polyfills enabled, Failed asserting that Array &0 (
    'FORM_C' => 4
    'FORM_D' => 2
    'FORM_KC' => 5
    'FORM_KD' => 3
    'NFC' => 4
    'NFD' => 2
    'NFKC' => 5
    'NFKD' => 3
    'NONE' => 1
) is identical to Array &0 (
    'FORM_C' => 16
    'FORM_D' => 4
    'FORM_KC' => 32
    'FORM_KC_CF' => 48
    'FORM_KD' => 8
    'NFC' => 16
    'NFD' => 4
    'NFKC' => 32
    'NFKC_CF' => 48
    'NFKD' => 8
    'NONE' => 2
).

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:37

17) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testIsNormalized
Polyfills enabled, Failed asserting that true is false.

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:53

18) Symfony\Polyfill\Tests\Intl\Normalizer\NormalizerTest::testNormalize
Polyfills enabled, Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'déjà 훈쇼™'
+'déjà 훈쇼™'

/home/km/projects/symfony-polyfill/tests/Intl/Normalizer/NormalizerTest.php:78

FAILURES!
Tests: 1100, Assertions: 364410, Failures: 18, Skipped: 12.

Unsilenced deprecation notices (5)

  5x: Normalizer::NONE is obsolete with ICU 56 and above and will be removed in later PHP versions
    3x in NormalizerTest::testNormalize from Symfony\Polyfill\Tests\Intl\Normalizer
    2x in NormalizerTest::testIsNormalized from Symfony\Polyfill\Tests\Intl\Normalizer

Legacy deprecation notices (136)

There's also a Debian bug report at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911832

[nicolas-grekas]

Closing as the CI is green on 7.3, which means this has been fixed - correct?

[vudaltsov]

@nicolas-grekas , I can confirm the problem for Mac. Tests fail for php 7.3.2 and pass fine for 7.1.25.
For instance, Normalizer constants values changed.

7.1.25

php > echo phpversion('intl');
1.1.0
php > var_dump((new \ReflectionClass(Normalizer::class))->getConstants());
array(9) {
  ["NONE"]=>
  int(1)
  ["FORM_D"]=>
  int(2)
  ["NFD"]=>
  int(2)
  ["FORM_KD"]=>
  int(3)
  ["NFKD"]=>
  int(3)
  ["FORM_C"]=>
  int(4)
  ["NFC"]=>
  int(4)
  ["FORM_KC"]=>
  int(5)
  ["NFKC"]=>
  int(5)
}

7.3.2

php > echo phpversion('intl');
7.3.2
php > var_dump((new \ReflectionClass(Normalizer::class))->getConstants());
array(11) {
  ["NONE"]=>
  int(2)
  ["FORM_D"]=>
  int(4)
  ["NFD"]=>
  int(4)
  ["FORM_KD"]=>
  int(8)
  ["NFKD"]=>
  int(8)
  ["FORM_C"]=>
  int(16)
  ["NFC"]=>
  int(16)
  ["FORM_KC"]=>
  int(32)
  ["NFKC"]=>
  int(32)
  ["FORM_KC_CF"]=>
  int(48)
  ["NFKC_CF"]=>
  int(48)
}

Now trying to investigate...

[vudaltsov]

I got it.

For php < 7.3: https://github.com/php/php-src/blob/782bbe722d90d8743f84da712d709a319699bfd6/ext/intl/normalizer/normalizer.h#L24

For php 7.3: https://github.com/php/php-src/blob/3fa88e0ce0ffd9f63672afe114158a07a0204e21/ext/intl/normalizer/normalizer.h#L22

[legoktm]

I can still reproduce the failures on Debian unstable (PHP 7.3.2-3). I don't know why they don't fail on travis-ci.

[vudaltsov]

@legoktm , here's my explanation.

• php < 7.3 + any ICU and php >= 7.3 + ICU < 56 work the same way.
• php >= 7.3 + ICU >= 56 works differently, see https://github.com/php/php-src/blob/3fa88e0ce0ffd9f63672afe114158a07a0204e21/ext/intl/normalizer/normalizer.h#L35 (this is why Normalizer constants have different values).

Now if you have php 7.3 and ICU 63 like I do, then tests fail.
Travis does not update icu regularly, see symfony/symfony#14259 and travis-ci/travis-ci#3616. That's why it does not fail on 7.3 yet.

[vudaltsov]

@nicolas-grekas , let's reopen this issue?
I am ready to try to fix it.

[nicolas-grekas]

Sure, thanks!

[vudaltsov]

@nicolas-grekas , actually PHP: nightly fails (https://travis-ci.org/symfony/polyfill/jobs/490568453) because of the ICU version, because php: nightly builds on travis checkout the latest ICU version (travis-ci/travis-ci#3616 (comment)).

So we now we know the problem and the testing tool! I will continue working on this.

[legoktm]

Ooh, nice find @nicolas-grekas. Thanks for working on this :-)

[vudaltsov]

@nicolas-grekas , @legoktm , I finally made tests fail. See #171