Linkification Data files and tooling

[macchiati]

https://github.com/unicode-org/properties/issues/507
See also https://github.com/unicode-org/unicode-reports/pull/247

Code to generate the property files and test data for UTS 58

• Incorporates changes from the PAG
• Also some wording updates
• The Link_Email property is also modified to be narrower. The reasons for this are:
  • If we have to make changes later, it is less disruptive to broaden the character set than to narrow it.
  • The non-ASCII are less commonly supported currently
  • I went with the identifiers from UAX 31, modified by what is valid in the ASCII ranges for the local-part:
    • \p{XID_Continue}
    • [\p{block=basic_latin}-\p{Cc}] // ASCII
    • -[\u0020 ; : " ( ) [ ] @ \ < >] // email exclusions from ASCII

See also the related spec changes in https://github.com/unicode-org/unicode-reports/pull/247

[macchiati]

TODO:

1. Update data file & test data generator to match rev 1 draft 5.
2. Add newer test data from ICANN.
3. Change the data file folder to /Public/<version>/linkification/
4. Change the filename SerializationTest.txt to FormattingTest.txt
5. Create a unicodetools PR with the code, get it reviewed & merged.

[markusicu]

Review of only the data files so far.

[markusicu]

I only skimmed chunks of the Java code.

[macchiati]

Added a note about the + in queries in minimal escaping:
https://unicode-org.github.io/unicode-reports/pull/247/tr58/tr58.html#minimal-escaping-algorithm

[markusicu]

One more comment for now, and there is at least one of the previous comment without discussion. I am also going over your discussion doc.

[markusicu]

for later: These files have trailing spaces now on empty comment lines, from the JOIN_N_HASH in the generator. We should clean that up at some point.