Merge pull request #428 from migueldeicaza/wcspec

For a while, we had a bit of a hodgepodge of attempts to measuring characters
properly. And I discovered ucs-detect [1] along with a spec for how terminal
emulators should measure unicode [2].

This patch implements that behavior.

Unicode width now comes from UCD properties + EastAsianWidth W/F ranges and
valid emoji‑variation sequences, and printing/combining now respects width ‑1/0
and only widens on valid VS16 (witha spacer cell) while dropping invalid
zero‑widths. We had a discussion last week on GitHub about some of these
problems, and my band-aid worked for a couple of things, but was by no means
comphrenesive.

[1] https://ucs-detect.readthedocs.io/intro.html
[2] https://wcwidth.readthedocs.io/en/latest/specs.html

The improvements are in:

https://gist.github.com/migueldeicaza/a81dc7fd5a31f10cd24b2ba60b28a083

See the bug for before/after screenshots.

This fixes #427

Author: migueldeicaza

Makefile

@@ -6,6 +6,9 @@ TOOLCHAINS=org.swift.573202201171a
 all:
 	echo nothing defined by default
 
+regen-unicode-width:
+	python3 scripts/regen_unicode_width_data.py
+
 jazzy:
 	(cd TerminalApp; DYLD_FALLBACK_LIBRARY_PATH=/Users/miguel/opt/anaconda3/lib/ jazzy --clean --author "Miguel de Icaza" --author_url https://tirania.org/ --github_url https://github.com/migueldeicaza/SwiftTerm --github-file-prefix https://github.com/migueldeicaza/SwiftTerm/tree/master --module-version 1.0 --module SwiftTerm --root-url https://migueldeicaza.github.io/SwiftTerm/ --output ../docs --build-tool-arguments -scheme,MacTerminal,-project,MacTerminal.xcodeproj)