Summary
slopsift fails to analyze a markdown file when emoji sit inside list items, reporting slopsift/runtime-error: "Document region markdown:list-item:N has an invalid UTF-16 source range." The CLI exits with code 2 in --format json mode on the larger original file; the minimal repro below surfaces the same runtime error as a diagnostic finding.
Reproduction
Save this as repro.md (269 bytes; the crash is length-sensitive, shortening either sentence makes it pass):
* 🚫 **DO NOT fertilize for 4–6 weeks.** Damaged roots are sensitive to fertilizer salts, which will burn them.
* 💧 **Watch your watering.** Do not let it sit in standing water. Water it only when the top inch or two of soil dries out (depending on the plant's species).
Run:
slopsift --rulepack ai-style --rulepack reader-first repro.md
Actual output:
repro.md
1:1 error SlopSift could not analyze this file: Document region markdown:list-item:1 has an invalid UTF-16 source range. slopsift/runtime-error
Expected: normal findings for the file (removing the two emoji yields ordinary lint results).
Analysis
- Both emoji (
🚫 U+1F6AB, 💧 U+1F4A7) are astral characters: one code point, two UTF-16 code units. Each one makes a code-point-based offset drift one unit behind the true UTF-16 offset. This points at region math that counts positions in code points but slices or validates in UTF-16 units.
- The region index in the error also drifts. On the original 4.2KB document this repro was minimized from, the error blamed
markdown:list-item:36 while the file contains only 30 list items.
- The crash needs alignment: 20 other emoji-bearing documents in the same corpus lint fine. It appears a rule's range (likely
sentence-load on the long second item) has to land after the drift for the range to become invalid.
Environment
- slopsift 0.8.2
- Node v22 (also reproduced on v24.10.0), macOS (Darwin 25.5.0)
Found while running a fix-loop pipeline over generated documents; reproduces deterministically.
Summary
slopsiftfails to analyze a markdown file when emoji sit inside list items, reportingslopsift/runtime-error: "Document region markdown:list-item:N has an invalid UTF-16 source range." The CLI exits with code 2 in--format jsonmode on the larger original file; the minimal repro below surfaces the same runtime error as a diagnostic finding.Reproduction
Save this as
repro.md(269 bytes; the crash is length-sensitive, shortening either sentence makes it pass):Run:
Actual output:
Expected: normal findings for the file (removing the two emoji yields ordinary lint results).
Analysis
🚫U+1F6AB,💧U+1F4A7) are astral characters: one code point, two UTF-16 code units. Each one makes a code-point-based offset drift one unit behind the true UTF-16 offset. This points at region math that counts positions in code points but slices or validates in UTF-16 units.markdown:list-item:36while the file contains only 30 list items.sentence-loadon the long second item) has to land after the drift for the range to become invalid.Environment
Found while running a fix-loop pipeline over generated documents; reproduces deterministically.