PEP-696 specialisation tests from CPython fail

[AlexWaygood]

We've fixed a number of issues with our PEP-696 backport in recent days, and we're now a lot closer to CPython's implementation on Python 3.13. We're still missing 5 specialisation-related tests from CPython's test suite, however, and they all fail using the typing_extensions implementation on Python 3.12:

Missing tests

diff --git a/src/test_typing_extensions.py b/src/test_typing_extensions.py
index fa04e59..0ab1e7f 100644
--- a/src/test_typing_extensions.py
+++ b/src/test_typing_extensions.py
@@ -6402,6 +6402,26 @@ class TypeVarLikeDefaultsTests(BaseTestCase):
         class A(Generic[Unpack[Ts]]): ...
         Alias = Optional[Unpack[Ts]]
 
+    def test_typevartuple_specialization(self):
+        T = TypeVar("T")
+        Ts = TypeVarTuple('Ts', default=Unpack[Tuple[str, int]])
+        self.assertEqual(Ts.__default__, Unpack[Tuple[str, int]])
+        class A(Generic[T, Unpack[Ts]]): ...
+        self.assertEqual(A[float].__args__, (float, str, int))
+        self.assertEqual(A[float, range].__args__, (float, range))
+        self.assertEqual(A[float, *tuple[int, ...]].__args__, (float, *tuple[int, ...]))
+
+    def test_typevar_and_typevartuple_specialization(self):
+        T = TypeVar("T")
+        U = TypeVar("U", default=float)
+        Ts = TypeVarTuple('Ts', default=Unpack[Tuple[str, int]])
+        self.assertEqual(Ts.__default__, Unpack[Tuple[str, int]])
+        class A(Generic[T, U, Unpack[Ts]]): ...
+        self.assertEqual(A[int].__args__, (int, float, str, int))
+        self.assertEqual(A[int, str].__args__, (int, str, str, int))
+        self.assertEqual(A[int, str, range].__args__, (int, str, range))
+        self.assertEqual(A[int, str, *tuple[int, ...]].__args__, (int, str, *tuple[int, ...]))
+
     def test_no_default_after_typevar_tuple(self):
         T = TypeVar("T", default=int)
         Ts = TypeVarTuple("Ts")
@@ -6487,6 +6507,34 @@ class TypeVarLikeDefaultsTests(BaseTestCase):
         a4 = Callable[[Unpack[Ts]], T]
         self.assertEqual(a4.__args__, (Unpack[Ts], T))
 
+    def test_paramspec_specialization(self):
+        T = TypeVar("T")
+        P = ParamSpec('P', default=[str, int])
+        self.assertEqual(P.__default__, [str, int])
+        class A(Generic[T, P]): ...
+        self.assertEqual(A[float].__args__, (float, (str, int)))
+        self.assertEqual(A[float, [range]].__args__, (float, (range,)))
+
+    def test_typevar_and_paramspec_specialization(self):
+        T = TypeVar("T")
+        U = TypeVar("U", default=float)
+        P = ParamSpec('P', default=[str, int])
+        self.assertEqual(P.__default__, [str, int])
+        class A(Generic[T, U, P]): ...
+        self.assertEqual(A[float].__args__, (float, float, (str, int)))
+        self.assertEqual(A[float, int].__args__, (float, int, (str, int)))
+        self.assertEqual(A[float, int, [range]].__args__, (float, int, (range,)))
+
+    def test_paramspec_and_typevar_specialization(self):
+        T = TypeVar("T")
+        P = ParamSpec('P', default=[str, int])
+        U = TypeVar("U", default=float)
+        self.assertEqual(P.__default__, [str, int])
+        class A(Generic[T, P, U]): ...
+        self.assertEqual(A[float].__args__, (float, (str, int), float))
+        self.assertEqual(A[float, [range]].__args__, (float, (range,), float))
+        self.assertEqual(A[float, [range], int].__args__, (float, (range,), int))

Here are the test failures if I add those tests to our suite:

(main)⚡ % python test_typing_extensions.py   ~/dev/typing_extensions/src
.................................................................................................................................................................s..s..s...........................................................................................................................................................................................E.E...EE...F..........................................s.........................
======================================================================
ERROR: test_paramspec_and_typevar_specialization (__main__.TypeVarLikeDefaultsTests.test_paramspec_and_typevar_specialization)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/alexw/dev/typing_extensions/src/test_typing_extensions.py", line 6534, in test_paramspec_and_typevar_specialization
    self.assertEqual(A[float].__args__, (float, (str, int), float))
                     ~^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 398, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1082, in _generic_class_getitem
    params = prepare(cls, params)
             ^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1036, in _paramspec_prepare_subst
    raise TypeError(f"Too few arguments for {alias}")
TypeError: Too few arguments for <class '__main__.TypeVarLikeDefaultsTests.test_paramspec_and_typevar_specialization.<locals>.A'>

======================================================================
ERROR: test_paramspec_specialization (__main__.TypeVarLikeDefaultsTests.test_paramspec_specialization)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/alexw/dev/typing_extensions/src/test_typing_extensions.py", line 6515, in test_paramspec_specialization
    self.assertEqual(A[float].__args__, (float, (str, int)))
                     ~^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 398, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1082, in _generic_class_getitem
    params = prepare(cls, params)
             ^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1036, in _paramspec_prepare_subst
    raise TypeError(f"Too few arguments for {alias}")
TypeError: Too few arguments for <class '__main__.TypeVarLikeDefaultsTests.test_paramspec_specialization.<locals>.A'>

======================================================================
ERROR: test_typevar_and_paramspec_specialization (__main__.TypeVarLikeDefaultsTests.test_typevar_and_paramspec_specialization)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/alexw/dev/typing_extensions/src/test_typing_extensions.py", line 6524, in test_typevar_and_paramspec_specialization
    self.assertEqual(A[float].__args__, (float, float, (str, int)))
                     ~^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 398, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1082, in _generic_class_getitem
    params = prepare(cls, params)
             ^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1036, in _paramspec_prepare_subst
    raise TypeError(f"Too few arguments for {alias}")
TypeError: Too few arguments for <class '__main__.TypeVarLikeDefaultsTests.test_typevar_and_paramspec_specialization.<locals>.A'>

======================================================================
ERROR: test_typevar_and_typevartuple_specialization (__main__.TypeVarLikeDefaultsTests.test_typevar_and_typevartuple_specialization)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/alexw/dev/typing_extensions/src/test_typing_extensions.py", line 6420, in test_typevar_and_typevartuple_specialization
    self.assertEqual(A[int].__args__, (int, float, str, int))
                     ~^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 398, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1082, in _generic_class_getitem
    params = prepare(cls, params)
             ^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/.pyenv/versions/3.12.3/lib/python3.12/typing.py", line 1011, in _typevartuple_prepare_subst
    raise TypeError(f"Too few arguments for {alias};"
TypeError: Too few arguments for <class '__main__.TypeVarLikeDefaultsTests.test_typevar_and_typevartuple_specialization.<locals>.A'>; actual 1, expected at least 2

======================================================================
FAIL: test_typevartuple_specialization (__main__.TypeVarLikeDefaultsTests.test_typevartuple_specialization)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/alexw/dev/typing_extensions/src/test_typing_extensions.py", line 6410, in test_typevartuple_specialization
    self.assertEqual(A[float].__args__, (float, str, int))
AssertionError: Tuples differ: (<class 'float'>,) != (<class 'float'>, <class 'str'>, <class 'int'>)

Second tuple contains 2 additional elements.
First extra element 1:
<class 'str'>

- (<class 'float'>,)
+ (<class 'float'>, <class 'str'>, <class 'int'>)

----------------------------------------------------------------------
Ran 435 tests in 0.059s

FAILED (failures=1, errors=4, skipped=4)

I don't know if it's going to be possible to fix this easily in typing_extensions, or if we should try to do so before the next release.

[AlexWaygood]

Actually it looks like this is pretty doable

[AlexWaygood]

#397 implements the correct behaviour on Python >=3.11.1. I don't know if it's feasible to backport it on earlier versions of Python, or if we should just document that it's a known limitation that __args__ may be incorrect on lower versions of Python if you're using typevarlikes with defaults.

[JelleZijlstra]

I think we can leave this open in case someone wants to add support. We have some similar open issues around edge cases with ParamSpec.