Add | union and [...] grouping operators, matching YARD #1700

lsegal/yard#1700 proposes standardizing `|` as an explicit union
operator and `[...]` as a grouping construct for YARD type tags,
alongside the `&` intersection operator this branch already added for
solargraph#1229. Implementing the full syntax here so Solargraph's own
parser and the upstream proposal describe the same grammar, and so
`(A | B) & C` - previously only buildable by translating real RBS or
constructing an Intersection object directly, per the now-outdated
comment on the parentheses spec - has an actual tag-string form.

`|` binds looser than `&` (matching RBS's documented precedence) and,
inside a fixed-arity context (`Array(...)` tuples, or a generic type's
positional parameters), groups multiple types into a single slot
instead of splitting into separate positional arguments - the same
distinction `,` already makes there. In an implicit-union context
(Array<...>/Set<...>, hash key/value lists, the top-level list itself),
`|` and `,` land on the same result, since every comma-separated type
in those contexts is already unioned regardless of grouping.

`[...]` is the actual grouping construct - the only way to mark where
a union ends when it needs to be one conjunct of an intersection
(`[Foo | Bar] & Baz`). It's deliberately conservative about when it
opens: only at a fresh atom (blank base, not already nested in
<>/{}/()), otherwise `[`/`]` are ordinary characters - this matters for
quoted string-literal types like `"[]"`, which have no concept of
grouping and would otherwise crash self-typecheck against the real Dir
RBS core stub.

Also fixes the anonymous shorthand forms `<A>`, `(A)`, `{A=>B}` (typed
before this as an empty-name UniqueType) to default their name to
Array/Array/Hash respectively, per YARD #1700's third documented
change - so an anonymous form now behaves exactly like its named
equivalent, including for rooting.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019wfsRatLaRbxcsQ7ZVzifN

Author: apiology

lib/solargraph/complex_type.rb

@@ -538,12 +538,17 @@ def parse_type_string type_string, types, key_types
         point_stack = 0
         curly_stack = 0
         paren_stack = 0
+        bracket_stack = 0
         base = String.new
         subtype_string = String.new
         # conjuncts of an intersection type (`A & B`) seen so far in
-        # the segment currently being parsed
+        # the current `|`-disjunct of the segment being parsed
         # @type [Array<ComplexType>]
         conjuncts = []
+        # disjuncts of a union type (`A | B`) seen so far in the
+        # segment currently being parsed
+        # @type [Array<ComplexType, ComplexType::UniqueType>]
+        disjuncts = []
         # @param char [String]
         type_string&.each_char do |char|
           if char == '='
@@ -555,17 +560,8 @@ def parse_type_string type_string, types, key_types
               subtype_string += char
             elsif base.end_with?('=')
               raise ComplexTypeError, 'Invalid hash thing' unless key_types.nil?
-              # types.push ComplexType.new([UniqueType.new(base[0..-2].strip)])
-              # @sg-ignore Need to add nil check here
-              types.push close_intersection(conjuncts, UniqueType.parse(base[0..-2].strip, subtype_string))
-              # @todo this should either expand key_type's type
-              #   automatically or complain about not being
-              #   compatible with key_type's type in type checking
-              key_types = types
+              key_types = close_key_types(base, subtype_string, conjuncts, disjuncts, types)
               types = []
-              conjuncts = []
-              base.clear
-              subtype_string.clear
               next
             else
               raise ComplexTypeError, "Invalid close in type #{type_string}" if point_stack.zero?
@@ -576,66 +572,168 @@ def parse_type_string type_string, types, key_types
           elsif char == '{'
             curly_stack += 1
           elsif char == '}'
-            curly_stack -= 1
-            subtype_string += char
-            raise ComplexTypeError, "Invalid close in type #{type_string}" if curly_stack.negative?
+            curly_stack = close_bracket(curly_stack, subtype_string, char, type_string)
             next
           elsif char == '('
             paren_stack += 1
           elsif char == ')'
-            paren_stack -= 1
-            subtype_string += char
-            raise ComplexTypeError, "Invalid close in type #{type_string}" if paren_stack.negative?
+            paren_stack = close_bracket(paren_stack, subtype_string, char, type_string)
             next
-          elsif char == '&' && top_level?(point_stack, curly_stack, paren_stack)
-            conjuncts.push ComplexType.new([UniqueType.parse(base.strip, subtype_string.strip)])
+          elsif char == '[' &&
+                (bracket_stack.positive? ||
+                 (base.strip.empty? && point_stack.zero? && curly_stack.zero? && paren_stack.zero?))
+            # Only a fresh atom (blank base, not already nested in
+            # <>/{}/()) can start a `[...]` group - matching
+            # finish_atom's own precondition. Otherwise `[` is just an
+            # ordinary character, e.g. part of a quoted string literal
+            # type like `"[]"`, which has no concept of grouping.
+            bracket_stack += 1
+          elsif char == ']' && bracket_stack.positive?
+            bracket_stack = close_bracket(bracket_stack, subtype_string, char, type_string)
+            next
+          elsif char == '&' && top_level?(point_stack, curly_stack, paren_stack, bracket_stack)
+            conjuncts.push ComplexType.new([finish_atom(base, subtype_string)])
+            base.clear
+            subtype_string.clear
+            next
+          elsif char == '|' && top_level?(point_stack, curly_stack, paren_stack, bracket_stack)
+            disjuncts.push close_intersection(conjuncts, finish_atom(base, subtype_string))
+            conjuncts = []
             base.clear
             subtype_string.clear
             next
-          elsif char == ',' && top_level?(point_stack, curly_stack, paren_stack)
-            # types.push ComplexType.new([UniqueType.new(base.strip, subtype_string.strip)])
-            types.push close_intersection(conjuncts, UniqueType.parse(base.strip, subtype_string.strip))
+          elsif char == ',' && top_level?(point_stack, curly_stack, paren_stack, bracket_stack)
+            disjuncts.push close_intersection(conjuncts, finish_atom(base, subtype_string))
+            types.push close_disjunction(disjuncts)
             conjuncts = []
+            disjuncts = []
             base.clear
             subtype_string.clear
             next
           end
-          if top_level?(point_stack, curly_stack, paren_stack)
+          if top_level?(point_stack, curly_stack, paren_stack, bracket_stack)
             base.concat char
           else
             subtype_string.concat char
           end
         end
-        if point_stack != 0 || curly_stack != 0 || paren_stack != 0
+        if point_stack != 0 || curly_stack != 0 || paren_stack != 0 || bracket_stack != 0
           raise ComplexTypeError,
                 "Unclosed subtype in #{type_string}"
         end
-        # types.push ComplexType.new([UniqueType.new(base, subtype_string)])
-        types.push close_intersection(conjuncts, UniqueType.parse(base.strip, subtype_string.strip))
+        disjuncts.push close_intersection(conjuncts, finish_atom(base, subtype_string))
+        types.push close_disjunction(disjuncts)
         [types, key_types]
       end
 
+      # Decrements the stack counter for a closing `}`/`)`/`]` and
+      # appends it to the pending subtype substring.
+      #
+      # @param stack [Integer]
+      # @param subtype_string [String]
+      # @param char [String]
+      # @param type_string [String, nil]
+      # @return [Integer] the decremented stack counter
+      def close_bracket stack, subtype_string, char, type_string
+        stack -= 1
+        subtype_string << char
+        raise ComplexTypeError, "Invalid close in type #{type_string}" if stack.negative?
+        stack
+      end
+
+      # Closes the key-list portion of a `Hash{K=>V}` split (the base
+      # ending in `=` marks the boundary) and returns the types parsed
+      # so far to be stashed as the eventual key_types, leaving
+      # conjuncts/disjuncts/base/subtype_string cleared for the value
+      # list that follows.
+      #
+      # @todo this should either expand key_type's type automatically
+      #   or complain about not being compatible with key_type's type
+      #   in type checking
+      #
+      # @param base [String]
+      # @param subtype_string [String]
+      # @param conjuncts [Array<ComplexType>]
+      # @param disjuncts [Array<ComplexType, ComplexType::UniqueType>]
+      # @param types [Array<ComplexType::UniqueType, ComplexType>]
+      # @return [Array<ComplexType::UniqueType, ComplexType>] the key_types
+      def close_key_types base, subtype_string, conjuncts, disjuncts, types
+        # @sg-ignore Need to add nil check here
+        disjuncts.push close_intersection(conjuncts, finish_atom(base[0..-2], subtype_string))
+        types.push close_disjunction(disjuncts)
+        conjuncts.clear
+        disjuncts.clear
+        base.clear
+        subtype_string.clear
+        types
+      end
+
       # @param point_stack [Integer]
       # @param curly_stack [Integer]
       # @param paren_stack [Integer]
+      # @param bracket_stack [Integer]
       # @return [Boolean]
-      def top_level? point_stack, curly_stack, paren_stack
-        point_stack.zero? && curly_stack.zero? && paren_stack.zero?
+      def top_level? point_stack, curly_stack, paren_stack, bracket_stack
+        point_stack.zero? && curly_stack.zero? && paren_stack.zero? && bracket_stack.zero?
       end
 
-      # Wraps a just-parsed unique type together with any pending
-      # intersection conjuncts (types seen so far in this segment,
+      # Resolves one type atom - either an ordinary named type (`base`
+      # plus its optional `<...>`/`(...)`/`{...}` parameter substring),
+      # or a standalone `[...]` grouping with no leading name, used to
+      # override the default order of operations (e.g. `[Foo | Bar] &
+      # Baz`, where `[...]` is the only way to mark where the union
+      # ends). A bracket group's content is parsed the same way any
+      # other parameter substring is - recursively, via
+      # ComplexType.parse - and its result substituted directly, since
+      # it can itself be a multi-item union (or an intersection).
+      #
+      # @param base [String]
+      # @param subtype_string [String]
+      # @return [ComplexType::UniqueType, ComplexType]
+      def finish_atom base, subtype_string
+        base = base.strip
+        subtype_string = subtype_string.strip
+        if base.empty? && subtype_string.start_with?('[')
+          raise ComplexTypeError, "Unclosed bracket group in #{subtype_string}" unless subtype_string.end_with?(']')
+          return ComplexType.new(ComplexType.parse(subtype_string[1..-2], partial: true))
+        end
+        UniqueType.parse(base, subtype_string)
+      end
+
+      # Wraps a just-parsed atom together with any pending
+      # intersection conjuncts (types seen so far in this disjunct,
       # separated by `&`) into a single UniqueType. Each conjunct is
       # a ComplexType (see UniqueType::Intersection), so the final
       # parsed type is promoted to a single-item ComplexType too.
       #
       # @param conjuncts [Array<ComplexType>]
-      # @param final_type [ComplexType::UniqueType]
-      # @return [ComplexType::UniqueType]
+      # @param final_type [ComplexType::UniqueType, ComplexType]
+      # @return [ComplexType::UniqueType, ComplexType]
       def close_intersection conjuncts, final_type
         return final_type if conjuncts.empty?
         UniqueType::Intersection.new(conjuncts + [ComplexType.new([final_type])])
       end
+
+      # Collapses the disjuncts of a union type (`A | B`) seen so far
+      # in the segment currently being parsed into a single value to
+      # push into the enclosing types/subtypes list - a bare type when
+      # there was only one (the common case, `|` never used), or a
+      # real multi-item ComplexType union otherwise. This is also
+      # exactly what a top-level `,` in an already-implicit-union
+      # context (Array<...>, Set<...>, hash key/value lists, the
+      # top-level types list itself) reduces to, since each of those
+      # contexts flattens every comma-separated type into one union
+      # regardless of how it's grouped here - so `,` and `|` land on
+      # the same result there, matching RBS's own tag design.
+      #
+      # @param disjuncts [Array<ComplexType, ComplexType::UniqueType>]
+      # @return [ComplexType::UniqueType, ComplexType]
+      # @sg-ignore #first is only nil for an empty array, and this is
+      #   never called with one
+      def close_disjunction disjuncts
+        return disjuncts.first if disjuncts.length == 1
+        ComplexType.new(disjuncts)
+      end
     end
 
     VOID = ComplexType.parse('void')

lib/solargraph/complex_type/unique_type.rb

@@ -13,6 +13,13 @@ class UniqueType
 
       attr_reader :all_params, :subtypes, :key_types
 
+      # @type [Hash{String => String}]
+      ANONYMOUS_NAME_BY_STARTING_TAG = {
+        '{' => 'Hash',
+        '(' => 'Array',
+        '<' => 'Array'
+      }.freeze
+
       # Create a UniqueType with the specified name and an optional substring.
       # The substring is the parameter section of a parametrized type, e.g.,
       # for the type `Array<String>`, the name is `Array` and the substring is
@@ -24,6 +31,11 @@ class UniqueType
       # @return [UniqueType]
       def self.parse name, substring = '', make_rooted: nil
         raise ComplexTypeError, "Illegal prefix: #{name}" if name.start_with?(':::')
+        # Anonymous shorthand - `<A>`, `(A)`, `{A=>B}` - omits the
+        # leading type name, defaulting it to Array or Hash. Resolved
+        # before the rooted/can_root_name? check below so an anonymous
+        # `<A>` behaves exactly like the equivalent `Array<A>`.
+        name = ANONYMOUS_NAME_BY_STARTING_TAG.fetch(substring[0]) if name.empty? && !substring.empty?
         if name.start_with?('::')
           name = name[2..]
           rooted = true