Skip to content

Commit f0833c3

Browse files
authored
Fix bracket/brace map corruption from Ruby 3.0+ pattern matching deconstruction (#1671)
# Description In Ruby 3.0+ pattern matching, bracket-based and braced patterns fire scanner events for their delimiters but do not fire the same parser events as ordinary array indexing or hash literals. YARD had no handlers for these pattern-match parser events, so entries were pushed to the bracket/brace maps but never consumed. Since those maps are read FIFO/LIFO by later expressions, the stale entries produced garbled source ranges on the next array indexing or hash literal after a pattern match. Three event handlers are added: - **`on_aryptn`**: Array patterns (`Bar['key', v]`, `Bar[a, b]`, bare `[a, b]`) fire `on_aryptn` instead of `on_aref`. The handler pops from `@map[:lbracket]` and shifts from `@map[:aref]`, mirroring what `on_aref` would have done. - **`on_fndptn`**: Find patterns (`Bar[*, v, *]`) fire `on_fndptn` instead of `on_aref`. Same bracket map cleanup. - **`on_hshptn`**: Braced hash patterns (`{ key: val }`) fire `on_hshptn` instead of `on_hash`, leaving a stale entry in `@map[:lbrace]`. To distinguish braced patterns from bare hash patterns (`key: val`, which fire no brace scanner events), `on_rbrace` is overridden to track closing brace positions in `@map[:rbrace]`. `on_hshptn` cleans up both maps only when `@map[:rbrace]` confirms a closing brace was scanned, using `||=` guards on both map accesses for consistency with the rest of the map handling code. `visit_event` is updated to drain `@map[:rbrace]` when consuming a `@map[:lbrace]` entry, keeping the maps in sync for normal hash literals and brace blocks (including the case where a braced pattern appears inside a brace block). Fixes #1547 # Completed Tasks - [x] I have read the [Contributing Guide][contrib]. - [x] The pull request is complete (implemented / written). - [x] Git commits have been cleaned up (squash WIP / revert commits). - [x] I wrote tests and ran `bundle exec rake` locally (if code is attached to PR). [contrib]: https://github.com/lsegal/yard/blob/main/CONTRIBUTING.md ----------------- # Description Describe your pull request and problem statement here. # Completed Tasks - [ ] I have read the [Contributing Guide][contrib]. - [ ] The pull request is complete (implemented / written). - [ ] Git commits have been cleaned up (squash WIP / revert commits). - [ ] I wrote tests and ran `bundle exec rake` locally (if code is attached to PR). [contrib]: https://github.com/lsegal/yard/blob/main/CONTRIBUTING.md
2 parents dac5f8b + 8a0ee42 commit f0833c3

2 files changed

Lines changed: 199 additions & 0 deletions

File tree

lib/yard/parser/ruby/ruby_parser.rb

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,7 @@ def visit_event(node)
244244
sstart = child_node.source_range.first
245245
else
246246
lstart, sstart = *(map ? map.pop : [lineno, @ns_charno - 1])
247+
(@map[:rbrace] ||= []).shift if map && MAPPINGS[node.type] == :lbrace
247248
end
248249

249250
raise "Cannot determine start of node #{node} around #{file}:#{lineno}" if lstart.nil? || sstart.nil?
@@ -336,6 +337,7 @@ def add_token(token, data)
336337
undef on_aref_field
337338
undef on_lbracket
338339
undef on_rbracket
340+
undef on_rbrace
339341
undef on_string_literal
340342
undef on_lambda
341343
undef on_unary
@@ -371,6 +373,20 @@ def on_hash(*args)
371373
visit_event AstNode.new(:hash, args.first || [])
372374
end
373375

376+
# Ruby 3.0+ pattern matching: braced hash patterns ({key: val} syntax) fire
377+
# on_lbrace and on_rbrace scanner events. The corresponding parser event is
378+
# on_hshptn (not on_hash), so we must clean up the brace maps to prevent stale
379+
# entries from corrupting source ranges of later hash literals and brace blocks.
380+
# Bare hash patterns (key: val without braces) fire no brace scanner events, so
381+
# we only clean up when @map[:rbrace] confirms a closing brace was scanned.
382+
def on_hshptn(*args)
383+
if (@map[:rbrace] ||= []).any?
384+
(@map[:lbrace] ||= []).pop
385+
@map[:rbrace].shift
386+
end
387+
AstNode.new(:hshptn, args)
388+
end
389+
374390
def on_bare_assoc_hash(*args)
375391
AstNode.new(:list, args.first)
376392
end
@@ -422,6 +438,27 @@ def on_array(other)
422438
node
423439
end
424440

441+
# Ruby 3.0+ pattern matching: array patterns (SomeClass[a, b]) and find patterns
442+
# (SomeClass[*pre, val, *post]) use [...] brackets, which fire on_lbracket and
443+
# on_rbracket scanner events. The corresponding parser events are on_aryptn/on_fndptn
444+
# (not on_aref), so we must clean up the bracket maps to prevent stale entries from
445+
# corrupting source ranges of later array indexing expressions.
446+
def on_aryptn(*args)
447+
(@map[:lbracket] ||= []).pop
448+
(@map[:aref] ||= []).shift
449+
# Source range is intentionally not set; no handler is registered for
450+
# pattern-match nodes, so they produce no documentation output.
451+
AstNode.new(:aryptn, args)
452+
end
453+
454+
def on_fndptn(*args)
455+
(@map[:lbracket] ||= []).pop
456+
(@map[:aref] ||= []).shift
457+
# Source range is intentionally not set; no handler is registered for
458+
# pattern-match nodes, so they produce no documentation output.
459+
AstNode.new(:fndptn, args)
460+
end
461+
425462
def on_lbracket(tok)
426463
(@map[:lbracket] ||= []) << [lineno, charno]
427464
visit_ns_token(:lbracket, tok, false)
@@ -432,6 +469,13 @@ def on_rbracket(tok)
432469
visit_ns_token(:rbracket, tok, false)
433470
end
434471

472+
# Maintained explicitly (unlike on_lbracket/on_rbracket) so on_hshptn can
473+
# distinguish braced from bare hash patterns in Ruby 3.0+ pattern matching.
474+
def on_rbrace(tok)
475+
(@map[:rbrace] ||= []) << [lineno, charno]
476+
visit_ns_token(:rbrace, tok, false)
477+
end
478+
435479
def on_dyna_symbol(sym)
436480
rng = if sym.source_range.to_a.size == 0 # rubocop:disable Style/ZeroLengthPredicate
437481
(sym.source_range.begin - 3)...sym.source_range.end

spec/parser/ruby/ruby_parser_spec.rb

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -679,6 +679,161 @@ def add(x) = x + 1
679679
end
680680
end if RUBY_VERSION >= '3.'
681681

682+
it "provides correct source range for aref after array pattern deconstruction" do
683+
code = <<-RUBY
684+
class Foo
685+
def check(obj)
686+
case obj
687+
in Bar["key", v]
688+
v
689+
end
690+
end
691+
692+
def fetch(h)
693+
h["result"]
694+
end
695+
end
696+
RUBY
697+
698+
parser = YARD::Parser::Ruby::RubyParser.new(code, nil)
699+
ast = parser.parse.root
700+
701+
aref_node = nil
702+
ast.traverse do |node|
703+
if node.type == :aref
704+
aref_node = node
705+
break
706+
end
707+
end
708+
709+
expect(aref_node).not_to be_nil
710+
expect(code[aref_node.source_range]).to eq('h["result"]')
711+
end if RUBY_VERSION >= '3.'
712+
713+
it "provides correct source range for aref after find pattern deconstruction" do
714+
code = <<-RUBY
715+
class Foo
716+
def check(obj)
717+
case obj
718+
in Bar[*, v, *]
719+
v
720+
end
721+
end
722+
723+
def fetch(h)
724+
h["result"]
725+
end
726+
end
727+
RUBY
728+
729+
parser = YARD::Parser::Ruby::RubyParser.new(code, nil)
730+
ast = parser.parse.root
731+
732+
aref_node = nil
733+
ast.traverse do |node|
734+
if node.type == :aref
735+
aref_node = node
736+
break
737+
end
738+
end
739+
740+
expect(aref_node).not_to be_nil
741+
expect(code[aref_node.source_range]).to eq('h["result"]')
742+
end if RUBY_VERSION >= '3.'
743+
744+
it "provides correct source range for hash literal after braced hash pattern deconstruction" do
745+
code = <<-RUBY
746+
class Foo
747+
def check(obj)
748+
case obj
749+
in { name: String }
750+
"matched"
751+
end
752+
end
753+
754+
def build
755+
{ name: "Alice" }
756+
end
757+
end
758+
RUBY
759+
760+
parser = YARD::Parser::Ruby::RubyParser.new(code, nil)
761+
ast = parser.parse.root
762+
763+
hash_node = nil
764+
ast.traverse do |node|
765+
if node.type == :hash
766+
hash_node = node
767+
break
768+
end
769+
end
770+
771+
expect(hash_node).not_to be_nil
772+
expect(code[hash_node.source_range]).to eq('{ name: "Alice" }')
773+
end if RUBY_VERSION >= '3.'
774+
775+
it "provides correct source range for aref after bare array pattern deconstruction" do
776+
code = <<-RUBY
777+
class Foo
778+
def check(obj)
779+
case obj
780+
in [a, b]
781+
a
782+
end
783+
end
784+
785+
def fetch(h)
786+
h["result"]
787+
end
788+
end
789+
RUBY
790+
791+
parser = YARD::Parser::Ruby::RubyParser.new(code, nil)
792+
ast = parser.parse.root
793+
794+
aref_node = nil
795+
ast.traverse do |node|
796+
if node.type == :aref
797+
aref_node = node
798+
break
799+
end
800+
end
801+
802+
expect(aref_node).not_to be_nil
803+
expect(code[aref_node.source_range]).to eq('h["result"]')
804+
end if RUBY_VERSION >= '3.'
805+
806+
it "provides correct source range for hash literal after bare hash pattern deconstruction" do
807+
code = <<-RUBY
808+
class Foo
809+
def check(obj)
810+
case obj
811+
in name: String
812+
"matched"
813+
end
814+
end
815+
816+
def build
817+
{ name: "Alice" }
818+
end
819+
end
820+
RUBY
821+
822+
parser = YARD::Parser::Ruby::RubyParser.new(code, nil)
823+
ast = parser.parse.root
824+
825+
hash_node = nil
826+
ast.traverse do |node|
827+
if node.type == :hash
828+
hash_node = node
829+
break
830+
end
831+
end
832+
833+
expect(hash_node).not_to be_nil
834+
expect(code[hash_node.source_range]).to eq('{ name: "Alice" }')
835+
end if RUBY_VERSION >= '3.'
836+
682837
it "provides correct range for `next` statement following `def` _symbol_" do
683838
code = <<-RUBY
684839
foo do

0 commit comments

Comments
 (0)