Adjust local variable presence to start after assignment, not before #864

apiology · 2025-04-01T18:24:33Z

Currently local variables are considered present within their assignment with the types of their assignments:

# a is considered as an integer in the next line
a = 123

However, in reality, a name is created, but assigned nil in its initial assignment code until the assignment statement is complete:

a = a + 1 # NoMethodError: undefined method `+' for nil:NilClass

(Note that we don't get a NameError: undefined local variable or method a' for main:Object`, so the variable has been created but not assigned while evaluating the rhs)

Currently, if you ask Solargraph to infer the position of 'a' in `a = 123', it gives Integer, which seems premature.

The current behavior may also seem innocuous, but it's getting in the way of my approach to #856, so I'd like to back off and instead have a local variable assignment's presence start on the next statement. Up until that point, it will have the same type it had before the statement. (nil would probably be better if it had no previous value, but I'm not sure that's worth the effort to emulate)

apiology · 2025-04-01T18:26:13Z

The big question in my mind - can we live with the impact this will have on the LSP world? If not, I might come back with a proposal that adds a 'human visibility' concept which is maybe different than 'machine visibility' :)

castwide · 2025-04-05T13:35:12Z

The big question in my mind - can we live with the impact this will have on the LSP world? If not, I might come back with a proposal that adds a 'human visibility' concept which is maybe different than 'machine visibility' :)

This is probably fine on the LSP side. My main concern is that messages like textDocument/hover should still provide the inferred type of a variable assignment. So far so good:

I'm going to test a little more locally, but I haven't seen any red flags yet.

castwide · 2025-04-07T16:00:36Z

👍

…astwide#864) * Adjust local variable presence to start after assignment, not before * Add regression test around assignment in return position * Fix assignment visibility code, which relied on bad asgn semantics

* Document a log level env variable * Fix logger reference * Fix env var name * Populate location information from RBS files (#768) * Populate location information from RBS files The 'rbs' gem maps the location of different definitions to the relevant point in the RGS files themselves - this change provides the ability to jump into the right place in those files to see the type definition via the LSP. * Prefer source location in language server * Resolve merge issue * Fix Path vs String type error * Consolidate parameter handling into Pin::Callable (#844) * Consolidate parameter handling into Pin::Closure * Clarify clobbered variable names * Fix bug in to_rbs, add spec, then fix new bug found after running spec * Catch one more Signature.new to translate from strict typechecking * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Use Pin::Callable type in args_node.rb * Select String#each_line overload with mandatory vs optional arg info * Adjust local variable presence to start after assignment, not before (#864) * Adjust local variable presence to start after assignment, not before * Add regression test around assignment in return position * Fix assignment visibility code, which relied on bad asgn semantics * Resolve params from ref tags (#872) * Resolve params from ref tags * Resolve ref tags with namespaces * Fix merge issue * RuboCop fixes * Add @sg-ignore * Linting * Linting fix * Linting fix --------- Co-authored-by: Fred Snyder <[email protected]>

* Document a log level env variable * Fix logger reference * Fix env var name * Populate location information from RBS files (castwide#768) * Populate location information from RBS files The 'rbs' gem maps the location of different definitions to the relevant point in the RGS files themselves - this change provides the ability to jump into the right place in those files to see the type definition via the LSP. * Prefer source location in language server * Resolve merge issue * Fix Path vs String type error * Consolidate parameter handling into Pin::Callable (castwide#844) * Consolidate parameter handling into Pin::Closure * Clarify clobbered variable names * Fix bug in to_rbs, add spec, then fix new bug found after running spec * Catch one more Signature.new to translate from strict typechecking * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Use Pin::Callable type in args_node.rb * Select String#each_line overload with mandatory vs optional arg info * Adjust local variable presence to start after assignment, not before (castwide#864) * Adjust local variable presence to start after assignment, not before * Add regression test around assignment in return position * Fix assignment visibility code, which relied on bad asgn semantics * Resolve params from ref tags (castwide#872) * Resolve params from ref tags * Resolve ref tags with namespaces * Fix merge issue * RuboCop fixes * Add @sg-ignore * Linting * Linting fix * Linting fix --------- Co-authored-by: Fred Snyder <[email protected]>

* Enable strict type checking in CI * Add 'solargraph method_pin' command for debugging ```sh $ SOLARGRAPH_ASSERTS=on bundle exec solargraph method_pin --rbs 'RuboCop::AST::ArrayNode#values' def values: () -> Array $ bundle exec solargraph help method_pin Usage: solargraph method_pin [PATH] Options: [--rbs], [--no-rbs], [--skip-rbs] # Output the pin as RBS # Default: false [--typify], [--no-typify], [--skip-typify] # Output the calculated return type of the pin from annotations # Default: false [--probe], [--no-probe], [--skip-probe] # Output the calculated return type of the pin from annotations and inference # Default: false [--stack], [--no-stack], [--skip-stack] # Show entire stack by including definitions in superclasses # Default: false Describe a method pin $ ``` * RuboCop and Solargraph fixes * Linting fix * Add spec * Fix spec * Allow newer RBS gem versions, exclude incompatible ones (#995) * Allow newer RBS gem versions This allow users to upgrade to recent Tapioca versions. Tapioca now requires newish versions of the spoom gem, which depends on 4.x pre-releases of the rbs gem. * Add RBS version to test matrix * Add exclude rule * Move to 3.6.1 * Look for external requires before cataloging bench (#1021) * Look for external requires before cataloging bench It seems like sync_catalog will go through the motions but not actually load pins from gems here due to passing an empty requires array to ApiMap. I'm sure those requires get pulled in eventually, but we go through at least one catalog cycle without it happening. Found while trying to test a different issue but not being able to get completions from a gem in a spec. * Ensure backport is pre-cached * Remove Library#folding_ranges (#904) * Complain in strong type-checking if an @sg-ignore line is not needed (#1011) * Complain in strong type-checking if an @sg-ignore line is not needed * Fix return type * Fix spec * Linting/coverage fixes * Coverage fix * Document a log level env variable (#894) * Document a log level env variable * Fix logger reference * Fix env var name * Populate location information from RBS files (#768) * Populate location information from RBS files The 'rbs' gem maps the location of different definitions to the relevant point in the RGS files themselves - this change provides the ability to jump into the right place in those files to see the type definition via the LSP. * Prefer source location in language server * Resolve merge issue * Fix Path vs String type error * Consolidate parameter handling into Pin::Callable (#844) * Consolidate parameter handling into Pin::Closure * Clarify clobbered variable names * Fix bug in to_rbs, add spec, then fix new bug found after running spec * Catch one more Signature.new to translate from strict typechecking * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Use Pin::Callable type in args_node.rb * Select String#each_line overload with mandatory vs optional arg info * Adjust local variable presence to start after assignment, not before (#864) * Adjust local variable presence to start after assignment, not before * Add regression test around assignment in return position * Fix assignment visibility code, which relied on bad asgn semantics * Resolve params from ref tags (#872) * Resolve params from ref tags * Resolve ref tags with namespaces * Fix merge issue * RuboCop fixes * Add @sg-ignore * Linting * Linting fix * Linting fix --------- Co-authored-by: Fred Snyder <[email protected]> * Fix hole in type checking evaluation (#1009) * Fix hole in type checking evaluation The call to `bar()` in `[ bar('string') ].compact` is not currently type-checked due to a missed spot in `NodeMethods#call_nodes_from(node)` * Avoid over-reporting call issues * Fix annotation * Add nocov markers around unreachable area * Fix a coverage issue * Cover more paths in type checking * Fix a code coverage issue * Drop blank line * Ratchet Rubocop todo * Fix missing coverage * Improve typechecking error message (#1014) If we know the target of an unresolved method call, include it in the error message. * Internal strict type-checking fixes (#1013) * Internal strict type-checking fixes * Add annotation * Add annotation * Add @sg-ignores for now * Reproduce and fix a ||= (or-asgn) evaluation issue (#1017) * Reproduce and fix a ||= (or-asgn) evaluation issue * Fix linting issue * Define closure for Pin::Symbol, for completeness (#1027) This isn't used anywhere to my knowledge, but it makes sense to think of symbols as being in the global namespace, helps guarantee that closure is always available on a pin, and of course keeps the assert happy ;) * Fix 'all!' config to reporters (#1018) * Fix 'all!' config to reporters Solargraph found the type error here! * Linting fixes * Fix DocMap.all_rbs_collection_gems_in_memory return type (#1037) * Fix RuboCop linting errors in regular expressions (#1038) * Fix RuboCop linting errors in regular expressions * Continue on rubocop_todo errors * Move configuration * Continue on undercover errors * Resolve class aliases via Constant pins (#1029) * Resolve class aliases via Constant pins This also eliminates the need for Parser::NodeMethods as a searately defined class. * Resolve merge issues * Resolve Solargraph strong complaint * Add @sg-ignore * Fix RuboCop issues * Drop unused method * Ratchet .rubocop_todo.yml * Speed-up LSP completion response times (#1035) * Improve performance of resolve_method_aliases method - Add indexed lookups for methods and aliases by name - Cache ancestor traversal to avoid repeated computations - Separate regular pins from aliases for more efficient processing - Replace linear search with direct indexed method lookup - Add fast path for creating resolved alias pins without individual lookups Generated with Claude Code * Update .rubocop_todo.yml * Fix typechecking - get_method_stack order `get_method_stack` returns the following order for `Enumerable#select`: - master branch: => ["Enumerable#select", "Kernel#select"] - current branch: => ["Kernel#select", "Enumerable#select"] * Avoid redundant indexing methods_by_name loop through ancestors and rely on store.get_path_pins * RuboCop todo update * Try rbs collection update before specs * RuboCop fixes * Add @sg-ignores * Fix typo * Exclude problematic combinations on Ruby head * Fix indentation * method_pin -> pin, hide command * Fix type * Make 'self' types concrete while checking arguments * strict -> strong * Also change default in Rakefile * RuboCop todo file stability To avoid merge conflicts and contributors having to deal with non-intuitive RuboCop todo changes: * Lock down development versions of RuboCop and plugins so that unrelated PRs aren't affected by newly implemented RuboCop rules. * Exclude rule entirely if more than 5 files violate it today, so that PRs are less likely to cause todo file changes unless they are specifically targeted at cleanup. * Clarify guidance on RuboCop todo file in CI error message. * Fix to hopefully ensure guidance always appears in CI error message. * Fix merge issue * Add spec * Work around strong typechecking issue * Pull in overcommit fix * Add spec * Add --references flag (superclass for now) * Add another @sg-ignore * Catch up with .rubocop_todo.yml * Add another @sg-ignore * Tolerate case statement in specs * Drop broken 'namespaces' method These unused methods call into ApiMap::Index#namespaces, which does not exist. * Rerun rubocop todo * Rebuild rubocop todo file * Force build * Restore * install -> update with rbs collection * Try Ruby 3.2 * Update solargraph * Re-add bundle install * Drop MATRIX_SOLARGRAPH_VERSION * Drop debugging changes * Update expectations from master branch * Update rubocop todo * Fix merge failure * [regression] Fix issue resolving mixins under same namespace * Prevent recursion via caching mechanism * Linting fix, ignore rubocop-yard issue pending yard PR merge * [regression] Fix resolution in deep YARD namespace hierarchies YARD-parsed namespaces weren't correctly setting their gates, leading to unresolved types from methods. * Linting * Fix merge * Resolve constants in references Also replaces 'include' logic with call to ApiMap::Constants Fixes #1099 * Linting * Ratchet RuboCop * Fix lack of parameters in include types * Bug fixes * Add actual type for Mutexes * Allow more valid method pin paths * RuboCop fix * Linting * Enable solargraph-rspec tests * Fix rspec gems specs * Closure merging fixes 1) Prefer closures with more gates, to maximize compatibility 2) Look at basename of location, to make choice consistent * Drop incidental requirement * [regression] Fix resolution of a nested type case Found in the Solargraph::Pin::DelegatedMethod class in strong typechecking in apiology#12 * Fix example name * Add another unmerged-Yard-PR issue * Fix merge issue * Ratchet RuboCop * Drop unneeded @sg-ignores * Drop unneeded @sg-ignore * Drop another @sg-ignore * Annotation fixes for strong typechecking * Add @sg-ignores for issues covered by future PRs * Add @sg-ignores for issues covered by future PRs * Reproduce solargraph-rspec rspec failure * Trigger build * Fix new bundler issue * Debug * Turn off warning diagnostics in CLI * Debug * Fix pin merging bug * Add @sg-ignore * Add @sg-ignore * Pull in solargraph-rspec fixes from another branch * Drop @sg-ignores * Add @sg-ignore * Drop @sg-ignore * Use consistent ruby versions for typechecking * Avoid ignoring valid pins in typechecking An overly-conservative check in type_checker.rb was suppressing later warnings on method parameters with no default values * Fix newly found type issue * Annotations needed after #1130 * Annotations needed after #1130 --------- Co-authored-by: Fred Snyder <[email protected]> Co-authored-by: Lekë Mula <[email protected]>

…1126) * Add 'solargraph method_pin' command for debugging ```sh $ SOLARGRAPH_ASSERTS=on bundle exec solargraph method_pin --rbs 'RuboCop::AST::ArrayNode#values' def values: () -> Array $ bundle exec solargraph help method_pin Usage: solargraph method_pin [PATH] Options: [--rbs], [--no-rbs], [--skip-rbs] # Output the pin as RBS # Default: false [--typify], [--no-typify], [--skip-typify] # Output the calculated return type of the pin from annotations # Default: false [--probe], [--no-probe], [--skip-probe] # Output the calculated return type of the pin from annotations and inference # Default: false [--stack], [--no-stack], [--skip-stack] # Show entire stack by including definitions in superclasses # Default: false Describe a method pin $ ``` * RuboCop and Solargraph fixes * Linting fix * Add spec * Fix spec * Allow newer RBS gem versions, exclude incompatible ones (#995) * Allow newer RBS gem versions This allow users to upgrade to recent Tapioca versions. Tapioca now requires newish versions of the spoom gem, which depends on 4.x pre-releases of the rbs gem. * Add RBS version to test matrix * Add exclude rule * Move to 3.6.1 * Look for external requires before cataloging bench (#1021) * Look for external requires before cataloging bench It seems like sync_catalog will go through the motions but not actually load pins from gems here due to passing an empty requires array to ApiMap. I'm sure those requires get pulled in eventually, but we go through at least one catalog cycle without it happening. Found while trying to test a different issue but not being able to get completions from a gem in a spec. * Ensure backport is pre-cached * Remove Library#folding_ranges (#904) * Complain in strong type-checking if an @sg-ignore line is not needed (#1011) * Complain in strong type-checking if an @sg-ignore line is not needed * Fix return type * Fix spec * Linting/coverage fixes * Coverage fix * Document a log level env variable (#894) * Document a log level env variable * Fix logger reference * Fix env var name * Populate location information from RBS files (#768) * Populate location information from RBS files The 'rbs' gem maps the location of different definitions to the relevant point in the RGS files themselves - this change provides the ability to jump into the right place in those files to see the type definition via the LSP. * Prefer source location in language server * Resolve merge issue * Fix Path vs String type error * Consolidate parameter handling into Pin::Callable (#844) * Consolidate parameter handling into Pin::Closure * Clarify clobbered variable names * Fix bug in to_rbs, add spec, then fix new bug found after running spec * Catch one more Signature.new to translate from strict typechecking * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Introduce Pin::Callable * Use Pin::Callable type in args_node.rb * Select String#each_line overload with mandatory vs optional arg info * Adjust local variable presence to start after assignment, not before (#864) * Adjust local variable presence to start after assignment, not before * Add regression test around assignment in return position * Fix assignment visibility code, which relied on bad asgn semantics * Resolve params from ref tags (#872) * Resolve params from ref tags * Resolve ref tags with namespaces * Fix merge issue * RuboCop fixes * Add @sg-ignore * Linting * Linting fix * Linting fix --------- Co-authored-by: Fred Snyder <[email protected]> * Fix hole in type checking evaluation (#1009) * Fix hole in type checking evaluation The call to `bar()` in `[ bar('string') ].compact` is not currently type-checked due to a missed spot in `NodeMethods#call_nodes_from(node)` * Avoid over-reporting call issues * Fix annotation * Add nocov markers around unreachable area * Fix a coverage issue * Cover more paths in type checking * Fix a code coverage issue * Drop blank line * Ratchet Rubocop todo * Fix missing coverage * Improve typechecking error message (#1014) If we know the target of an unresolved method call, include it in the error message. * Internal strict type-checking fixes (#1013) * Internal strict type-checking fixes * Add annotation * Add annotation * Add @sg-ignores for now * Reproduce and fix a ||= (or-asgn) evaluation issue (#1017) * Reproduce and fix a ||= (or-asgn) evaluation issue * Fix linting issue * Define closure for Pin::Symbol, for completeness (#1027) This isn't used anywhere to my knowledge, but it makes sense to think of symbols as being in the global namespace, helps guarantee that closure is always available on a pin, and of course keeps the assert happy ;) * Fix 'all!' config to reporters (#1018) * Fix 'all!' config to reporters Solargraph found the type error here! * Linting fixes * Fix DocMap.all_rbs_collection_gems_in_memory return type (#1037) * Fix RuboCop linting errors in regular expressions (#1038) * Fix RuboCop linting errors in regular expressions * Continue on rubocop_todo errors * Move configuration * Continue on undercover errors * Resolve class aliases via Constant pins (#1029) * Resolve class aliases via Constant pins This also eliminates the need for Parser::NodeMethods as a searately defined class. * Resolve merge issues * Resolve Solargraph strong complaint * Add @sg-ignore * Fix RuboCop issues * Drop unused method * Ratchet .rubocop_todo.yml * Speed-up LSP completion response times (#1035) * Improve performance of resolve_method_aliases method - Add indexed lookups for methods and aliases by name - Cache ancestor traversal to avoid repeated computations - Separate regular pins from aliases for more efficient processing - Replace linear search with direct indexed method lookup - Add fast path for creating resolved alias pins without individual lookups Generated with Claude Code * Update .rubocop_todo.yml * Fix typechecking - get_method_stack order `get_method_stack` returns the following order for `Enumerable#select`: - master branch: => ["Enumerable#select", "Kernel#select"] - current branch: => ["Kernel#select", "Enumerable#select"] * Avoid redundant indexing methods_by_name loop through ancestors and rely on store.get_path_pins * RuboCop todo update * Try rbs collection update before specs * RuboCop fixes * Add @sg-ignores * Fix typo * Exclude problematic combinations on Ruby head * Fix indentation * method_pin -> pin, hide command * Fix type * RuboCop todo file stability To avoid merge conflicts and contributors having to deal with non-intuitive RuboCop todo changes: * Lock down development versions of RuboCop and plugins so that unrelated PRs aren't affected by newly implemented RuboCop rules. * Exclude rule entirely if more than 5 files violate it today, so that PRs are less likely to cause todo file changes unless they are specifically targeted at cleanup. * Clarify guidance on RuboCop todo file in CI error message. * Fix to hopefully ensure guidance always appears in CI error message. * Fix merge issue * Add --references flag (superclass for now) * Add another @sg-ignore * Catch up with .rubocop_todo.yml * Add another @sg-ignore * Tolerate case statement in specs * Rerun rubocop todo * Force build * Restore * install -> update with rbs collection * Try Ruby 3.2 * Update solargraph * Re-add bundle install * Drop MATRIX_SOLARGRAPH_VERSION * Drop debugging changes * Update expectations from master branch * Update rubocop todo * Fix merge failure * Allow more valid method pin paths * RuboCop fix * Linting * Reproduce solargraph-rspec rspec failure * Trigger build * Fix new bundler issue * Debug * Turn off warning diagnostics in CLI * Debug * Fix pin merging bug * Add @sg-ignore * Allow levels to be changed for typechecking rules in .solargraph.yml Suggested updates for https://solargraph.org/guides/configuration: 1) Change block in Defaults section to: ``` include: - "**/*.rb" exclude: - spec/**/* - test/**/* - vendor/**/* - ".bundle/**/*" require: [] domains: [] reporters: - rubocop - require_not_found formatter: rubocop: cops: safe except: [] only: [] extra_args: [] type_checker: rules: {} max_files: 5000 ``` 2) Add section 'formatter' (hadn't been added) Specify configuration for code formatters used in the LSP. Currently supports 'rubocop' as the subsection, with the following keys: * `cops`: Valid values: 'all', 'safe' * `except`: List of cops to pass to the --except argument in the RuboCop CLI * `only`: List of cops to pass to the --only argument in the RuboCop CLI * `extra_args`: Additional arguments to pass to the RuboCop CLI 3) Add section 'type_checker.rules' Override which rules are applied at a given typechecking level. Example: ``` type_checker: rules: validate_calls: typed # make 'typed' level more strict without # bringing in other rules ``` For the current list of typechecking rules, see [rules.rb](https://github.com/castwide/solargraph/blob/master/lib/solargraph/type_checker/rules.rb) in the Solargraph source. * Fix merge * Fix merge --------- Co-authored-by: Fred Snyder <[email protected]> Co-authored-by: Lekë Mula <[email protected]>

Adjust local variable presence to start after assignment, not before

a890c43

Add regression test around assignment in return position

db2cd70

apiology mentioned this pull request Apr 10, 2025

Add regression test around assignment in return position #887

Closed

apiology added 2 commits April 10, 2025 10:41

Merge remote-tracking branch 'castwide/master' into variable_visibility

77d0a63

Merge branch 'regression_test' into variable_visibility

329b42e

apiology marked this pull request as draft April 10, 2025 15:12

Fix assignment visibility code, which relied on bad asgn semantics

9677375

apiology marked this pull request as ready for review April 10, 2025 18:47

Merge remote-tracking branch 'castwide/master' into variable_visibility

9ba538f

castwide merged commit cc38f09 into castwide:master Apr 19, 2025
8 checks passed

castwide mentioned this pull request May 14, 2025

Clip#define returns variable pins #934

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Adjust local variable presence to start after assignment, not before #864

Adjust local variable presence to start after assignment, not before #864

Uh oh!

apiology commented Apr 1, 2025

Uh oh!

apiology commented Apr 1, 2025

Uh oh!

castwide commented Apr 5, 2025

Uh oh!

castwide commented Apr 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Adjust local variable presence to start after assignment, not before #864

Adjust local variable presence to start after assignment, not before #864

Uh oh!

Conversation

apiology commented Apr 1, 2025

Uh oh!

apiology commented Apr 1, 2025

Uh oh!

castwide commented Apr 5, 2025

Uh oh!

castwide commented Apr 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants