Skip to content

Only consider meta variables in ambiguous "any of" constraints #18986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

sterliakov
Copy link
Collaborator

@sterliakov sterliakov commented Apr 28, 2025

Sometimes our constraints builder needs to express "at least one of these constraints must be true". Sometimes it's trivial, but sometimes they have nothing in common - in that case we fall back to forgetting all of them and (usually) inferring Never.

This PR extends the fallback handling logic by one more rule: before giving up, we try restricting the constraints to meta variables only. This means that when we try to express T :> str || U :> str in the following setup:

from typing import TypeVar

T = TypeVar("T")
U = TypeVar("U")

class Foo(Generic[T]):
    def func(self, arg: T | U) -> None: ...

we won't ignore both and fall back to Never but will pick U :> str instead. The reason for this heuristic is that function-scoped typevars are definitely something we're trying to infer, while everything else is usually out of our control, any other type variable should stay intact.

There are other places where constraint builder may emit restrictions on type variables it does not control, handling them consistently everywhere is left as an exercise to the reader.

This isn't safe in general case - it might be that another typevar satisfies the constraints but the chosen one doesn't. However, this shouldn't make any existing inference worse: if we used to infer Never and it worked, then anything else should almost definitely work as well.

See the added testcase for motivation: currently mypy fails to handle Mapping.get with default without return type context when Mapping has a type variable as the second argument. https://mypy-play.net/?mypy=1.15.0&python=3.12&flags=strict&gist=2f9493548082e66b77750655d3a90218

This is a prerequisite of #18976 - that inference change makes the problem solved here occur more often.

This comment has been minimized.

This comment has been minimized.

@sterliakov sterliakov changed the title [experiment] Prefer "own" type vars (owned by current function) when building constraints Prefer "own" type vars (owned by current function) when building "any of" constraints Apr 28, 2025
@sterliakov sterliakov marked this pull request as ready for review April 28, 2025 02:31
@sterliakov sterliakov requested a review from ilevkivskyi April 28, 2025 16:25
@sterliakov
Copy link
Collaborator Author

@ilevkivskyi I understand no one wants to touch this inference logic to avoid regressions, but I think I found a very nice improvement in #18976 that closes a bunch of issues with a commonly used idiom, and this is the only prerequisite to fix one major regression caused by that. Could you have a look and say whether this idea makes sense at all?

@ilevkivskyi
Copy link
Member

Hm, although the general idea is correct (and moreover unfortunately there are places scattered around the code where we treat unrelated type variables incorrectly), I think the implementation may be simplified. Instead of what you are doing, can you try to filter away type variables that are not meta vars? I.e, try simply

[c for c in option if c.type_var.id.is_meta_var()]

as a fallback (you may need to handle cases when this returns all and/or none of the constraints).

@sterliakov
Copy link
Collaborator Author

...I'm feeling dumb now, I swear that was my very first attempt! This works and is definitely saner - perhaps I missed the empty result case initially.

Yes, I do see that non-meta typevars are mishandled here, but probably don't have enough energy to fix that.

Copy link
Contributor

github-actions bot commented May 5, 2025

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@sterliakov sterliakov changed the title Prefer "own" type vars (owned by current function) when building "any of" constraints Only consider meta variables in ambiguous "any of" constraints May 6, 2025
@sterliakov
Copy link
Collaborator Author

It didn't cause any primer regressions, the issues check is also clean (sterliakov/mypy-issues#32 - two changed issues; both improvements, but insufficient to consider them resolved). @ilevkivskyi ready for another round!

Copy link
Member

@ilevkivskyi ilevkivskyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG, thanks, just some minor suggestions.

def get(self, key: _K, default: Union[_V, _T]) -> Union[_V, _T]: ...

def check(mapping: Mapping[str, _T]) -> None:
ok1 = mapping.get("", "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a reveal_type(ok1) here.


# If normal logic didn't work, try excluding trivially unsatisfiable constraint (due to
# upper bounds) from each option, and comparing them again.
filtered_options = [filter_satisfiable(o) for o in options]
if filtered_options != options:
return any_constraints(filtered_options, eager=eager)

# Try harder: if that didn't work, try to strip typevars that aren't meta vars.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Try harder: if that didn't work, try to strip typevars that aren't meta vars.
# Try harder: if that didn't work, try to strip typevars that aren't meta vars.
# Note this is what we would always do, but unfortunately some callers may not
# set the meta var status correctly (for historical reasons), so we use this as
# a fallback only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants