Skip to content

Erase stray typevars in functools.partial generic #18954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion mypy/plugins/functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import mypy.plugin
import mypy.semanal
from mypy.argmap import map_actuals_to_formals
from mypy.erasetype import erase_typevars
from mypy.nodes import (
ARG_POS,
ARG_STAR2,
Expand Down Expand Up @@ -312,7 +313,11 @@ def handle_partial_with_callee(ctx: mypy.plugin.FunctionContext, callee: Type) -
special_sig="partial",
)

ret = ctx.api.named_generic_type(PARTIAL, [ret_type])
# Do not leak typevars from generic functions - they cannot be usable.
# Keep them in the wrapped callable, but avoid `partial[SomeStrayTypeVar]`
erased_ret_type = erase_typevars(ret_type, [tv.id for tv in fn_type.variables])

ret = ctx.api.named_generic_type(PARTIAL, [erased_ret_type])
ret = ret.copy_with_extra_attr("__mypy_partial", partially_applied)
if partially_applied.param_spec():
assert ret.extra_attrs is not None # copy_with_extra_attr above ensures this
Expand Down
68 changes: 68 additions & 0 deletions test-data/unit/check-functools.test
Original file line number Diff line number Diff line change
Expand Up @@ -658,3 +658,71 @@ def f(x: P):
# TODO: but this is incorrect, predating the functools.partial plugin
reveal_type(partial(x, "a")()) # N: Revealed type is "builtins.int"
[builtins fixtures/tuple.pyi]

[case testFunctoolsPartialTypeVarErasure]
from typing import Callable, TypeVar, Union
from typing_extensions import ParamSpec, TypeVarTuple, Unpack
from functools import partial

def use_int_callable(x: Callable[[int], int]) -> None:
pass
def use_func_callable(
x: Callable[
[Callable[[int], None]],
Callable[[int], None],
],
) -> None:
pass

Tc = TypeVar("Tc", int, str)
Tb = TypeVar("Tb", bound=Union[int, str])
P = ParamSpec("P")
Ts = TypeVarTuple("Ts")

def func_b(a: Tb, b: str) -> Tb:
return a
def func_c(a: Tc, b: str) -> Tc:
return a

def func_fn(fn: Callable[P, Tc], b: str) -> Callable[P, Tc]:
return fn
def func_fn_unpack(fn: Callable[[Unpack[Ts]], Tc], b: str) -> Callable[[Unpack[Ts]], Tc]:
return fn

# We should not leak stray typevars that aren't in scope:
use_int_callable(partial(func_b, b=""))
use_func_callable(partial(func_b, b=""))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this pass? I assume the erased return type is Union[int, str] and not Never...

Also could you add some reveal_type like use_func_callable(reveal_type(partial(func_b, b="")))

Copy link
Collaborator Author

@sterliakov sterliakov Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The erased return type is Any. erase_typevars replaces them with Any (special_form), not with upper bounds.

Adding reveal_type is a good idea, let's show the inferred generics explicitly. I don't even need use_*_callable then... no, let's keep both - subtype checks may do something different

Copy link
Collaborator

@A5rocks A5rocks Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should still have the use_*_callable to ensure things still work, at least in a few spots -- printing correctly doesn't necessarily mean the logic is right, e.g. maybe the plugin could mess up passing the partials to the use_*_callable functions (?).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And Any is really what we want. Replacing tvars with their upper bounds will cause another bunch of false positives whenever smth like def fn[T: str | int](x: T) -> T (hope I didn't mess up this ugly syntax) is used with partial.

use_int_callable(partial(func_c, b=""))
use_func_callable(partial(func_c, b=""))
use_int_callable(partial(func_fn, b="")) # E: Argument 1 to "use_int_callable" has incompatible type "partial[Callable[[VarArg(Any), KwArg(Any)], Any]]"; expected "Callable[[int], int]" \
# N: "partial[Callable[[VarArg(Any), KwArg(Any)], Any]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Callable[[VarArg(Any), KwArg(Any)], Any]]"
use_func_callable(partial(func_fn, b=""))
use_int_callable(partial(func_fn_unpack, b="")) # E: Argument 1 to "use_int_callable" has incompatible type "partial[Callable[[VarArg(Any)], Any]]"; expected "Callable[[int], int]" \
# N: "partial[Callable[[VarArg(Any)], Any]].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Callable[[VarArg(Any)], Any]]"
use_func_callable(partial(func_fn_unpack, b=""))

# But we should not erase typevars that aren't bound by function
# passed to `partial`:

def outer_b(arg: Tb) -> None:

def inner(a: Tb, b: str) -> Tb:
return a

def call(fn: Callable[[int], int]) -> int:
return fn(0)

call(partial(inner, b="")) # E: Argument 1 to "call" has incompatible type "partial[Tb]"; expected "Callable[[int], int]" \
# N: "partial[Tb].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], Tb]"

def outer_c(arg: Tc) -> None:

def inner(a: Tc, b: str) -> Tc:
return a

def call(fn: Callable[[int], int]) -> int:
return fn(0)

call(partial(inner, b="")) # E: Argument 1 to "call" has incompatible type "partial[str]"; expected "Callable[[int], int]" \
# N: "partial[str].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], str]"
[builtins fixtures/tuple.pyi]