Regex engine throws index out of range

[LGL-Ben]

Description

I'm encountering an IndexOutOfRangeException when using the following regex:

string r = "(?>(-*)+?-*)$";

Reproduction Steps

Use the above regular expression with Regex.Match.
Match against any non-zero length string.
A reproduction can be found here:
https://sharplab.io/#v2:C4LgTgrgdgNAJiA1AHwAIEYB0AVApgD2EwCVcBzCAGwEMwBRfABzFwGdWBLAeylZPIKYAstWABjABYAKAESYZMGVID8APikBaAFQBKRMu06AJDJ0BuIA
I removed as much from it as much as I could.

Expected behavior

The function should return without throwing an error.

Actual behavior

The function throws an IndexOutOfRangeException

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

It should be noted that I observed the same error thrown from multiple internal functions, as I removed parts of the expression to simplify it.

[kaiwanyawit-tw]

ah...can reproduce it here https://dotnetfiddle.net/biRoXs

using System;
using System.Text.RegularExpressions;

class Program
{
    static void Main()
    {
        string pattern = @"(?>(-*)+?-*)$";

        // Sample test strings
        string[] testStrings = {
            "",
            "-",
            "--",
            "---",
            "----",
            "abc---",
            "--abc",
            "abc",
            "--\n",
            "-----a",
            "----\n"
        };

        Regex regex = new Regex(pattern);

        Console.WriteLine($"Testing pattern: {pattern}\n");

        foreach (string input in testStrings)
        {
            bool isMatch = regex.IsMatch(input);
            Console.WriteLine($"Input: \"{input}\" => Match: {isMatch}");
        }
    }
}

[am11]

It's an Interpreter specific behavior. It doesn't repro with Compiled or Generated engines:

using System;
using System.Text.RegularExpressions;

var match = Patterns.Problem.Match("."); // works
Console.WriteLine("Generated: {0}", match);

Regex regex = new("(?>(-*)+?-*)$", RegexOptions.Compiled);
match = regex.Match("."); // works
Console.WriteLine("Compiled {0}", match);

match = Regex.Match(".","(?>(-*)+?-*)$"); // IndexOutOfRangeException
Console.WriteLine("Interpreter: {0}", match);

partial class Patterns
{
    [GeneratedRegex("(?>(-*)+?-*)$")]
    public static partial Regex Problem { get; }
}

[pra2892]

Based on the pattern (?>(-*)+?-*)$, it seems you're trying to match sequences of hyphens at the end of a string with specific backtracking behavior.

The combination of nested quantifiers inside an atomic group was causing the regex engine to get into an invalid state.

Here's the possible solution to avoid the IndexOutOfRangeException while maintaining similar matching behavior:

View Code Here ->

using System;
using System.Text.RegularExpressions;

class Program
{
    static void Main()
    {
        // Original problematic pattern
        try
        {
            Console.WriteLine("Testing original pattern:");
            var match1 = Regex.Match(".", "(?>(-*)+?-*)$");
            Console.WriteLine($"Match: {match1.Success}");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Exception: {ex.GetType().Name} - {ex.Message}");
        }

        // Best solution
        Console.WriteLine("\nTesting best solution:");
        var match2 = Regex.Match(".", "(?:-*)*$");
        Console.WriteLine($"Match: {match2.Success}");
        
        // Test with various inputs
        string[] testStrings = { ".", "a", "-", "---", "a---", "---a" };
        Console.WriteLine("\nTesting with various inputs:");
        foreach (var str in testStrings)
        {
            var match = Regex.Match(str, "(?:-*)*$");
            Console.WriteLine($"'{str}' => Match: {match.Success}, Value: '{match.Value}'");
        }
    }
}

[LGL-Ben]

(Irrelevant lore dump:)

Based on the pattern (?>(-*)+?-*)$, it seems you're trying to match sequences of hyphens at the end of a string with specific backtracking behavior.

While I appreciate the effort - I really do -, that was not what I was trying to do.

I was designing some super long regex. As one does...
And after finishing it, I tried to fold some logic in on itself to make it more concise. (Code golf, I guess.)

I ended up adding an atomic group somewhere at somepoint in the refactoring process, which gave me an unexpected error.
I investigated the error and to much surprise ended up finding it wasn't caused by my code.

So... I culled many lines of regex from that expression, until I arrrived at a crashing expression with as few characters as possible.
The matched dashes used to be many lines of subexpressions.

I had honestly expected to just get a "false" back at that point.
I wasn't expecting the Regex to match. I also wasn't expecting it to crash my code sandbox.

Again, thanks for providing a workaround. Very kind of you.

Kind regards,
Ben

[LGL-Ben]

As an interesting side-note:
I was creating the expression on regex101, originally.

There, removing the $ at the end produced an ArgumentException in the engine as well.
I couldn't repro that in sharplab, so I didn't report it.
I figure that's probably something wrong with that site.

[pra2892]

Thanks for the additional context!

It's fascinating how you discovered this edge case in the .NET regex engine.

What do you think, is it a bug in the .NET regex engine when it handles certain combinations of atomic groups and nested quantifiers?

Also after removing the $ anchor produced a different exception (ArgumentException) on regex101 but not in sharplab is interesting.

possibly regex101 might be using a different version of the .NET regex engine.

[LGL-Ben]

What do you think, is it a bug in the .NET regex engine when it handles certain combinations of atomic groups and nested quantifiers?

I'd assume so?

I can't speak to it for certain, however I couldn't replicate the error in expressions any shorter than that.
I tried removing the shy modifier '?'; I tried using other group constructs...
Something about this specific setup manages to get the engine into several invalid states.

I'll attempt to replicate the other Error Message tomorrow, no guarantees on success though.

[LGL-Ben]

Firstly, I'd like to add as information, that regex101 labels the regex 'flavor' for .Net as ".Net 7.0 (C#)"
Under the hood, it seems to me they download some static webassembly-packaged .dll to run .Net Regex.

I've tried extensively to understand the argument exception, but couldn't replicate it in a useful manner elsewhere.
I'd assume it's probably got a reason similar to the reported error, as many of the explicit errors in the runner's source are bounds checks.

It seems almost impressive to me that the snippet at the top causes an IndexOutOfRangeException, rather than being caught at one of the great many bounds checks strewn across the runner's source. Never tell me the odds.

[steveharter]

Appears to be interpreter-specific; moving to V10.

Other issues that throw IndexOutOfRangeException:
#111161
#58786