Commit 7becc4e
committed
Stop the lexer reading past the end of a byte_range
`rbs_next_char` ends the input when `byte_pos == end_pos`. `rbs_skip`
advances by a whole character, so a multibyte character starting before
`end_pos` and ending after it steps over the boundary and equality never
holds again — the lexer then reads to the end of the string.
Parser.parse_type('"Ωx" | Integer', byte_range: 0...2)
#=> Types::Union spanning 0...14, the whole input
Two bytes were requested and fourteen were read, and `require_eof: true`
does not catch it because the lexer really is at EOF by then. It needs a
character to straddle the boundary, so ASCII-only input never hits it.
Compare with `>=` so stepping over the boundary still ends the input.1 parent aa48bf2 commit 7becc4e
2 files changed
Lines changed: 25 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
122 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
123 | 125 | | |
124 | 126 | | |
125 | 127 | | |
| |||
132 | 134 | | |
133 | 135 | | |
134 | 136 | | |
135 | | - | |
| 137 | + | |
| 138 | + | |
136 | 139 | | |
137 | 140 | | |
138 | 141 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1029 | 1029 | | |
1030 | 1030 | | |
1031 | 1031 | | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
1032 | 1052 | | |
0 commit comments