Skip to content

Assert parser UTF-16 input invariant - #136

Open
bobzhang wants to merge 1 commit into
codex/utf16-scanner-indexingfrom
codex/parser-utf16-sanitize
Open

Assert parser UTF-16 input invariant#136
bobzhang wants to merge 1 commit into
codex/utf16-scanner-indexingfrom
codex/parser-utf16-sanitize

Conversation

@bobzhang

@bobzhang bobzhang commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Assert in Parser::new that parser input is well-formed UTF-16
  • Panic with a clear message on unpaired surrogate code units instead of silently repairing invalid input
  • Add black-box renderer tests documenting valid astral input and malformed-surrogate panic cases

Scope

This is the parser-boundary invariant slice extracted from #127. It intentionally avoids adding a public sanitizer API: the parser maintains the invariant that its MoonBit String input is well-formed, and malformed surrogate code-unit sequences are treated as programmer error from unsafe or host-side construction.

This PR is stacked after #135 because the earlier PRs fix the shared UTF-16 helpers, renderer handling, and valid-Unicode scanner indexing.

Dependency order: #132 -> #134 -> #135 -> this PR.

Validation

  • moon test src/cmark_html --target wasm-gc
  • moon check --warn-list +unnecessary_annotation
  • moon test
  • moon info

@bobzhang
bobzhang marked this pull request as ready for review June 13, 2026 12:44
@bobzhang
bobzhang force-pushed the codex/utf16-scanner-indexing branch from 4150901 to bcd19a6 Compare June 13, 2026 13:17
@bobzhang
bobzhang force-pushed the codex/parser-utf16-sanitize branch from 44905f3 to 38aeb8d Compare June 13, 2026 13:19
@bobzhang bobzhang changed the title Sanitize parser UTF-16 input Assert parser UTF-16 input invariant Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant