🧼 generate compact wcwidth tables (#49)

* ref(wcwidth): generate efficient wcwidth tables

* ref(wcwidth): deslop

* perf(wcwidth): add BMP filter + fast path, use single table for binary search

* ref(ci): add verify task

* deno fmt

* ref(wcwidth): generate clang-format friendly code

* fix(verify): handle --check flag

* deno lint

* ref(wcwidth): add datahash

* ref(wcwidth): correctness pass

* ref(wcwidth): performance pass

* ref(wcwidth): upgrade to unicode 17

* ref(wcwidth): regen

* fix format

* fix(render): measure non-printable codepoints as one cell

The draw path coerces wcwidth() == -1 to one cell and emits U+FFFD, but
the Clay measure callback skipped those codepoints entirely, so a
fit-sized box around text containing a control character or
noncharacter measured narrower than what the renderer draws. Mirror
draw_text in measure(): coerce -1 to 1, and substitute U+FFFD on UTF-8
decode failure instead of passing an uninitialized codepoint to
wcwidth().

* fix(wcwidth): treat surrogates as non-printable

utf8_decode() accepts CESU-8 surrogate encodings, and the Unicode 17
tables default unlisted codepoints to width 1, so a surrogate reaching
draw_text was emitted to the terminal as invalid UTF-8 (the old
termbox2 table returned -1 for U+D800..U+DFFF and the emit path
sanitized it to U+FFFD). Guard the surrogate range in the generated
wcwidth() so iswprint() rejects it and the emit path substitutes
U+FFFD again.

Also derive the generated header's Unicode version from the same
constant as the data URL; it was left saying 16.0 after the 17.0
upgrade.

Author: natemoo-re

.github/workflows/verify.yaml

@@ -34,8 +34,8 @@ jobs:
           # Restore to v2.x once v2.9.1 ships.
           deno-version: v2.8.3
 
-      - name: format
-        run: deno task fmt:check
+      - name: verify
+        run: deno task verify
 
       - name: lint
         run: deno lint

deno.json

@@ -6,7 +6,10 @@
     "fmt": "deno fmt && clang-format -i src/*.c src/*.h",
     "fmt:check": "deno fmt --check && clang-format --dry-run --Werror src/*.c src/*.h",
     "build:npm": "deno run -A tasks/build-npm.ts",
-    "bench": "deno run -A bench/mod.ts"
+    "bench": "deno run -A bench/mod.ts",
+    "gen-wcwidth": "deno run --allow-net --allow-write=src/wcwidth.c tasks/gen-wcwidth.ts",
+    "gen-wcwidth:check": "deno run --allow-net --allow-read=src/wcwidth.c tasks/gen-wcwidth.ts --check",
+    "verify": "deno task fmt:check && deno task gen-wcwidth:check"
   },
   "imports": {
     "@std/testing": "jsr:@std/testing@1",

src/clayterm.c

@@ -856,8 +856,13 @@ void measure(int ret, int txt) {
     int n = utf8_decode(&cp, p);
     if (n <= 0) {
       n = 1;
+      cp = 0xfffd;
     }
+    /* Mirror draw_text: non-printables render as one U+FFFD cell, so
+     * they must measure as one cell. */
     int cw = wcwidth(cp);
+    if (cw < 0)
+      cw = 1;
     if (cw > 0)
       w += cw;
     p += n;