IndexOutOfBoundsException parsing valid YAML whose size is equal to StreamReader's buffer size

[mkosmul]

What version of OpenRewrite are you using?

OpenRewrite 8.40.3 but issue is also present in main as of 2025-02-26

What is the smallest, simplest way to reproduce the problem?

The following test fails:

    @Test
    void parseFlowSequenceAtBufferBoundary() throws IOException {
        // May change over time in SnakeYaml, rendering this test fragile
        var snakeYamlEffectiveStreamReaderBufferSize = 1024 - 1;

        @Language("yml")
        var yaml = "a: " + "x".repeat(1000) + "\n" + "b".repeat(16) + ": []";
        assertEquals(snakeYamlEffectiveStreamReaderBufferSize - 1, yaml.lastIndexOf('['));

        rewriteRun(
            spec -> spec.recipe(new DeleteKey(".nonexistent","*")),
            yaml(yaml)
        );
    }

The recipe used doesn't matter, it's there just to trigger the YAML parser.

This test fails with the following exception:

Failed to parse sources or run recipe
java.lang.AssertionError: Failed to parse sources or run recipe
	at org.openrewrite.test.RewriteTest.lambda$defaultExecutionContext$14(RewriteTest.java:638)
	at org.openrewrite.test.RewriteTest$$Lambda$509/0x0000020b8117b730.accept(Unknown Source)
	at org.openrewrite.yaml.YamlParser.lambda$parseInputs$0(YamlParser.java:76)
	at org.openrewrite.yaml.YamlParser$$Lambda$549/0x0000020b8125f340.apply(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.openrewrite.test.RewriteTest.rewriteRun(RewriteTest.java:281)
	at org.openrewrite.test.RewriteTest.rewriteRun(RewriteTest.java:129)
	at org.openrewrite.yaml.YamlParserTest.parseFlowSequenceAtBufferBoundary(YamlParserTest.java:335)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.IndexOutOfBoundsException: Index 3 out of bounds for length 3
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
	at java.base/java.util.Objects.checkIndex(Objects.java:361)
	at java.base/java.util.ArrayList.get(ArrayList.java:427)
	at org.openrewrite.yaml.FormatPreservingReader.readStringFromBuffer(FormatPreservingReader.java:112)
	at org.openrewrite.yaml.YamlParser.parseFromInput(YamlParser.java:364)
	at org.openrewrite.yaml.YamlParser.lambda$parseInputs$0(YamlParser.java:70)
	... 16 more


Index 3 out of bounds for length 3
java.lang.IndexOutOfBoundsException: Index 3 out of bounds for length 3
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
	at java.base/java.util.Objects.checkIndex(Objects.java:361)
	at java.base/java.util.ArrayList.get(ArrayList.java:427)
	at org.openrewrite.yaml.FormatPreservingReader.readStringFromBuffer(FormatPreservingReader.java:112)
	at org.openrewrite.yaml.YamlParser.parseFromInput(YamlParser.java:364)
	at org.openrewrite.yaml.YamlParser.lambda$parseInputs$0(YamlParser.java:70)
	at org.openrewrite.yaml.YamlParser$$Lambda$549/0x000002389525f340.apply(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.openrewrite.test.RewriteTest.rewriteRun(RewriteTest.java:281)
	at org.openrewrite.test.RewriteTest.rewriteRun(RewriteTest.java:129)
	at org.openrewrite.yaml.YamlParserTest.parseFlowSequenceAtBufferBoundary(YamlParserTest.java:335)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

What happens is FormatPreservingReader.readStringFromBuffer() assumes the end mark of an event points to a character in the buffer. However, in the case of an opening bracket in a flow-style sequence ([), event.getEndMark() points past the character. This is fine if the sequence start character is in the middle of the buffer, but if it happens to be the last character in the buffer, the reader tries to access the character behind it, causing the IndexOutOfBoundsException.

What did you expect to see?

The test should pass as the recipe should successfully parse the YAML and not modify it.

What did you see instead?

Parsing the YAML failed.

What is the full stack trace of any errors you encountered?

See above.

Are you interested in contributing a fix to OpenRewrite?

Yes, I have a fix already and will open a PR shortly.