Pattern Matching

[AnthonyDGreen]

Some people have suggested to me that pattern matching doesn't feel like a VB feature. What they mean is "Pattern matching sounds like some advanced programming stuff". I disagree. I think pattern matching is very much so a VB feature. In fact, I suggest that most VB programmers, myself included, have spent most of their careers looking at data--be it DB records, or string input, or messages from one device or another--and testing the shape of that data against known patterns, separating structure from content, and extracting select content to fit our needs. That's what every use of the Like operator, the InStr function, and most If' and Select statements are really doing. They're just going about it in a rather tedious (imperative) way. Pattern Matching is a powerful feature we can add to VB to make the same kinds of tasks we've always been doing much more concise, readable, and declarative.

I propose we extend existing Case statements to support a new kind of clause, the Match or Matches clause (depending on your grammatical proclivities) of the form:

Case Match pattern [When expression]

Where pattern is one of:

The wildcard pattern
*
Matches all values, including null.

The variable pattern
identifier [?] [As typename]

• Declares a new variable with name identifier.
• If no type is specified will match any non-null value.
• If a type is specified the value much be of that type.
• If the ? modifier is used will match null values but will still fail if the value is a non-null value of another type than specified.
• Type of the variable is either the specified type is provided, or inferred from the value being matched if no type is provided, or nullable version in either case if the ? modifier is used.

The function pattern
expression ( [ pattern1 [, pattern2, ...] ] )

• Matches if the function resolved from evaluating expression returns true. This function may optionally provided one or more outputs, each of which is recursively matched against the corresponding specified pattern.

Example:

Select Case "1"
    Case Integer.TryParse(value) When value > 0
End Select

Order of execution in the above example is as follows:

• Integer.TryParse is called passing the value of the Select Case statement in as its first argument, a new variable named 'value' of the type of the second argument of the TryParse method is passed in as the second argument (which is ByRef).
• If the invocation of TryParse returns true, execution passes to the When clause. The expression value > 0 is evaluated as true.
• Execution then proceeds into the Case block.

The tuple pattern
( pattern1 [, pattern2, ...] )

• Matches a tuple of an arity equal to the number of patterns specified. Recursively matches each element of the tuple against the corresponding pattern.

However, the list doesn't end here. In fact, it's my belief that the tuple pattern is just the first example of a new trend in the language that with each new expression we add for constructing an object, there is a corresponding pattern for decomposing that object that mirrors the construction syntax. For example, one can imagine (and I have) patterns based on decomposing arrays based on the array literal syntax, decomposing strings based on the string interpolation syntax, and decomposing XML fragments based on the XML literal syntax. The JSON literal proposal (#101) also demonstrates this.

The entire Match (or Matches) clause succeeds if the specified pattern succeeds AND the expression provided in the When clause (if provided) evaluates as true. The expression may refer to variables introduced in pattern.

[AdamSpeight2008]

Another possible option is to extend the usage cases of Like.

[zspitz]

Pattern matching is absolutely a VB.NET feature, especially considering it already exists in limited form as part of Select Case -- To. Is <comparison operator> x etc.

I would suggest that any pattern matching syntax should have the following features:

• Uses the same syntax for patterns in a boolean-returning expression (introduced with the Matches keyword), and for patterns in a Case clause:

  Dim o As Object
  '...
  Dim flag As Boolean = o Matches String 'No identifier named String in current scope, so this is a type check
  
  Select Case o
      Case String
          Debug.Print("It's a String")
  End Select

  This enables using the same pattern matching syntax in If ... Then, Do ... While blocks and the like.

• Allows introducing variables into the child scope:

  Dim o As Object
  '...
  If o Matches s As String Then
      Debug.Print($"Length of string: {s.Length}")
  End If
  
  Select Case o
      Case s As String
          Debug.Print($"Length of string: {s.Length}")
  End Select
  
  'When clause is also a child scope:
  If o Matches s As String When s.Length > 5 Then
      Debug.Print("String is longer than 5 characters.")
  End If

• Allows various patterns, not just type-checking

  Dim i As Integer
  ' ...
  
  'patterns already valid in Select Case, applied to a theoretical PatternExpression
  'range pattern
  Dim flag As Boolean = i Matches 1 To 10
  
  'comparison pattern; Is is optional
  flag = i Matches Is > 47
  
  ' OR pattern, with range subpattern and comparison subpattern
  flag = i Matches 1 To 10, Is > 47

• Allows When clauses

  Dim i As Integer
  '...
  Dim flag As Boolean = i Matches Is > 47 When i Mod 2 = 0

• Allows nested patterns

  Dim i As Integer
  Select Case i
      ' Commonly referred to as the OR pattern, matches if any of the subpatterns matches
      Case > 105, < 0, 5 To 20
          Console.WriteLine("> 105, negative, or between 5 and 20")
  End Select

---

Proposed spec (using bits from the ANTLR-like grammer available from here):

PatternExpression
    : Pattern ('When' BooleanExpression)?
    ;

Pattern
    // patterns with subpatterns
    : Pattern ',' Pattern                    // OR pattern
    | '(' Pattern (',' Pattern)* ')'         // Tuple pattern
    | 'Not' Pattern                          // NOT pattern

    // patterns without subpatterns
    | '*'                                    // Matches any value, including Nothing
    | Literal                                // Matches a literal value
    | Identifier                             // Existing Identifier pattern -- when identifier already exists in scope; tests for value/reference equality
    | 'Of'? TypeName                         // Type check pattern -- when type name already exists in scope
    | Identifier ('As' TypeName)?            // Variable pattern; introduces a new variable in child scope
                                             // The type of the variable (if not specified) is the same as the subject of the pattern match
    | 'Is'? ComparisonOperator Expression    // Comparison pattern
    | 'Like' StringExpression                // Like pattern
    | Expression 'To' Expression             // Range pattern
    | Expression                             // Equality pattern -- value/reference equality test against Expression
    ;

// usage in Select Case` looks like this:
// replaces existing CaseStatement
CaseStatement
    : 'Case' PatternExpression StatementTerminator
      Block?
    ;

and theoretically, it could be used anywhere BooleanExpression is used; with the caveat that since the pattern is greedy, either a PatternExpression or a BooleanExpression can be used, but not both.

BooleanOrPatternExpression
    : BooleanExpression
    | PatternExpression
    ;

// If...Then..ElseIf blocks
BlockIfStatement
    : 'If' BooleanOrPatternExpression 'Then'? StatementTerminator
      Block?
      ElseIfStatement*
      ElseStatement?
      'End' 'If' StatementTerminator
    ;

ElseIfStatement
    : ElseIf BooleanOrPatternExpression 'Then'? StatementTerminator
      Block?
    ;

LineIfThenStatement
    : 'If' BooleanOrPatternExpression 'Then' Statements ( 'Else' Statements )? StatementTerminator
    ;

WhileStatement
    : 'While' BooleanOrPatternExpression StatementTerminator
      Block?
      'End' 'While' StatementTerminator
    ;

DoTopLoopStatement
    : 'Do' ( WhileOrUntil BooleanOrPatternExpression )? StatementTerminator
      Block?
      'Loop' StatementTerminator
    ;

// cannot introduce variables in to child scope here
DoBottomLoopStatement
    : 'Do' StatementTerminator
      Block?
      'Loop' WhileOrUntil BooleanOrPatternExpression StatementTerminator
    ;

ConditionalExpression
    : 'If' OpenParenthesis BooleanOrPatternExpression Comma Expression Comma Expression CloseParenthesis
    | 'If' OpenParenthesis Expression Comma Expression CloseParenthesis
    ;

Some additional points:

• The list of patterns is certainly not exhaustive.

• I've tried to incorporate the existing variations that Select Case allows.

• As written here, an identifier could resolve to one of three possibilities: an existing identifier in the parent scope, a type name, or a new variable introduced in the child scope; using the following algorithm:

  • If identifier exists in parent scope, match against parent scope identifier using value/reference equality
  • if identifier is a type name in parent scope, match subject against type
  • else, create a new variable in scope

  I think conflicts of this nature are rare enough that this algorithm is sufficient; while allowing optional syntax for forcing the type check pattern (Of identifier) or forcing the variable pattern (identifier As typename).
  But if this is too "clever" for VB.NET, then these parts of the pattern could be made non-optional. This would mean that all the patterns are inherently non-ambiguous; and would probably be easier on both the compiler, and people who are learning pattern matching.

• This spec doesn't prevent the variable pattern even when there is no child scope, e.g. Dim flag As Boolean = o Matches s As String, which is probably a bad thing. OTOH, the When clause is also a child scope.

• The Like pattern could incorporate interpolated strings, inspired by String patterns #140

• Allowing a When clause on each sub-pattern is a problem, because this:

  Dim x As Integer
  If x Matches 15, 20 To 25 When x Mod 2 = 0 Then

  would apply the When only to the 20 To 25 sub-pattern. How could it be applied to the entire pattern? If the standard VB.NET idiom for operation association is reserved here for the tuple pattern, then it could not be used for association. Therefore I think allowing a single When clause per pattern would have to suffice.

---

Related issues:

#307 , #191 -- When expressions
#304
#24
#160 -- Use of pattern matching syntax to define a BooleanExpression
#23 -- Could be covered by a combination of OR pattern and type-check pattern

Pattern matching as a BooleanExpression would obviate the need for a dedicated type-checking syntax (#277 @bandleader) and provide inferred typing on introduced variables (#172).

Specific patterns: #141, #140, #139

Pinging @KathleenDollard @AdamSpeight2008 @bandleader @gafter @AnthonyDGreen @ericmutta