Skip to content

[ty] Model walrus bindings from comprehensions#26466

Draft
charliermarsh wants to merge 34 commits into
mainfrom
charlie/issue-162-comprehension-walrus-scope
Draft

[ty] Model walrus bindings from comprehensions#26466
charliermarsh wants to merge 34 commits into
mainfrom
charlie/issue-162-comprehension-walrus-scope

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Jun 30, 2026

Copy link
Copy Markdown
Member

Summary

PEP 572 specifies that an assignment-expression target inside a comprehension binds in the scope containing the outermost comprehension. Before this change, we left the target inside the comprehension scope, so it was unresolved when referenced afterward. For example, these two cases raised false positives on main:

def first_comment(lines: list[str]):
    if any((comment := line).startswith("#") for line in lines):
        reveal_type(comment)  # str

def partial_sums(values: list[int]):
    total = 0
    sums = [total := total + value for value in values]
    reveal_type(total)  # int

This PR makes the walrus target visible in the containing scope while keeping its value expression in the comprehension scope, where it can reference iteration variables.

What's included vs. excluded

I went through several iterations of this change over the past few months, and each time they ballooned in scope. Here, I'm trying to intentionally limit the scope, so this section articulates cases that came up in Codex review but were explicitly rejected as out-of-scope.

We do not model the zero-iteration path or whether a generator expression has been consumed:

def empty_comprehension():
    items: list[int] = []
    [(last := item) for item in items]
    print(last)  # Runtime NameError; this PR treats `last` as bound.

def deferred_generator(items: list[int]):
    pending = ((last := item) for item in items)
    print(last)  # Runtime NameError until `pending` is consumed; modeled eagerly here.

We also conservatively promote exported values because later iterations can change them. We do not try to retain single-iteration literal precision:

[(last := 1) for _ in [0]]
reveal_type(last)  # int, not Literal[1]

We do not compute a fixed point for a walrus whose value changes type across repeated iterations. We currently infer the exported type from the first modeled iteration:

value = 0
[value := "" if isinstance(value, int) else 0 for _ in [0, 1]]
reveal_type(value)  # str; the runtime value after the second iteration is int
value.upper()  # Currently accepted; a fixed-point model would infer str | int

(This requires treating the comprehension body as a loop and iterating its use-def state to a fixed point.)

We also defer all the IDE-specific navigation, references, rename, and unused-binding behavior to a separate PR (#26476).

Closes astral-sh/ty#162.

@astral-sh-bot astral-sh-bot Bot added the ty Multi-file analysis & type inference label Jun 30, 2026
@astral-sh-bot

astral-sh-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 94.47%. The percentage of expected errors that received a diagnostic held steady at 89.19%. The number of fully passing files held steady at 95/134.

@astral-sh-bot

astral-sh-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
prefect 449.56MB 449.62MB +0.01% (62.52kB)
sphinx 167.06MB 167.08MB +0.01% (16.41kB)
trio 70.47MB 70.47MB +0.00% (2.18kB)
flake8 29.02MB 29.02MB +0.00% (1.16kB)

Significant changes

Click to expand detailed breakdown

prefect

Name Old New Diff Outcome
semantic_index 114.29MB 114.34MB +0.05% (52.73kB)
infer_scope_types_impl 30.19MB 30.19MB +0.02% (5.07kB)
Definition 20.48MB 20.48MB +0.02% (4.17kB)
infer_definition_types 50.04MB 50.04MB -0.00% (184.00B)
is_redundant_with_impl::interned_arguments 2.35MB 2.35MB +0.01% (176.00B)
UnionType 1.07MB 1.07MB +0.01% (128.00B)
infer_deferred_types 6.00MB 6.00MB +0.00% (120.00B)
Type<'db>::apply_specialization_inner_ 2.04MB 2.04MB +0.00% (104.00B)
analyze_non_terminal_call 1.78MB 1.78MB -0.00% (88.00B)
FunctionType<'db>::signature_ 2.56MB 2.56MB +0.00% (80.00B)
is_redundant_with_impl 1.31MB 1.31MB +0.01% (80.00B)
infer_expression_types_impl 38.01MB 38.01MB +0.00% (64.00B)
all_narrowing_constraints_for_expression 5.23MB 5.23MB +0.00% (64.00B)
infer_expression_type_impl 219.79kB 219.82kB +0.01% (32.00B)
StaticClassLiteral<'db>::implicit_attribute_inner_ 811.21kB 811.20kB -0.00% (16.00B)

sphinx

Name Old New Diff Outcome
semantic_index 37.37MB 37.39MB +0.04% (14.09kB)
infer_scope_types_impl 8.10MB 8.10MB +0.02% (1.38kB)
Definition 7.41MB 7.41MB +0.01% (1016.00B)
infer_expression_types_impl 14.89MB 14.89MB -0.00% (48.00B)
StaticClassLiteral<'db>::implicit_attribute_inner_ 605.77kB 605.73kB -0.01% (32.00B)
all_narrowing_constraints_for_expression 1.94MB 1.94MB +0.00% (32.00B)
analyze_non_terminal_call 683.03kB 683.05kB +0.00% (16.00B)
infer_expression_type_impl 223.78kB 223.77kB -0.01% (16.00B)
member_lookup_with_policy_inner 4.29MB 4.29MB -0.00% (16.00B)
Type<'db>::apply_specialization_inner_ 1.04MB 1.04MB +0.00% (8.00B)
infer_deferred_types 2.67MB 2.67MB +0.00% (8.00B)

trio

Name Old New Diff Outcome
semantic_index 17.73MB 17.73MB +0.01% (1.62kB)
Definition 3.62MB 3.62MB +0.01% (408.00B)
infer_scope_types_impl 2.40MB 2.41MB +0.01% (152.00B)
infer_expression_types_impl 4.42MB 4.42MB +0.00% (16.00B)

flake8

Name Old New Diff Outcome
semantic_index 7.86MB 7.86MB +0.01% (1016.00B)
infer_scope_types_impl 512.46kB 512.61kB +0.03% (152.00B)
Definition 1.81MB 1.81MB +0.00% (16.00B)

@astral-sh-bot

astral-sh-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

Lint rule Added Removed Changed
unresolved-reference 0 8 0
Total 0 8 0

Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.

Raw diff:

core (https://github.com/home-assistant/core)
- homeassistant/components/ista_ecotrend/sensor.py:277:47 error[unresolved-reference] Name `statistics_sum` used when not defined
- homeassistant/components/recorder/history/__init__.py:909:53 error[unresolved-reference] Name `prev_state` used when not defined
- homeassistant/components/recorder/history/__init__.py:925:49 error[unresolved-reference] Name `prev_state` used when not defined

mkosi (https://github.com/systemd/mkosi)
- mkosi/__init__.py:5249:80 error[unresolved-reference] Name `c` used when not defined
- mkosi/__init__.py:5254:79 error[unresolved-reference] Name `c` used when not defined
- mkosi/config.py:4776:41 error[unresolved-reference] Name `missing` used when not defined

steam.py (https://github.com/Gobot1234/steam.py)
- steam/_gc/client.py:56:90 error[unresolved-reference] Name `previous_state` used when not defined

xarray (https://github.com/pydata/xarray)
- xarray/core/extension_array.py:154:16 error[unresolved-reference] Name `ea_type` used when not defined

Full report with detailed diff (timing results)

@charliermarsh charliermarsh force-pushed the charlie/issue-162-comprehension-walrus-scope branch from 0c783d5 to 40e790a Compare June 30, 2026 14:18
@charliermarsh charliermarsh added the bug Something isn't working label Jun 30, 2026
@charliermarsh charliermarsh force-pushed the charlie/issue-162-comprehension-walrus-scope branch from cdb9361 to 1ce0f53 Compare June 30, 2026 19:05
charliermarsh added a commit that referenced this pull request Jun 30, 2026
## Summary

This separates constructing and registering a `Definition` from
recording it in the current scope's use-def state.
`push_additional_definition` continues to create and associate
AST-backed definitions, then delegates the existing binding,
declaration, capture, and try-context bookkeeping to
`record_definition`.

This is a behavior-neutral refactor split from #26466 so synthesized
definitions can reuse the recording path without mixing the mechanical
extraction into the comprehension-walrus change. The ty core and
semantic suites and repository hooks pass.
@charliermarsh charliermarsh force-pushed the charlie/issue-162-comprehension-walrus-scope branch from 1ce0f53 to d5e0dbf Compare June 30, 2026 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

walrus expressions in a comprehension scope "leak" into the comprehension's enclosing scope

1 participant