feat: split, sort, and deduplicate curated dictionary #594
Conversation
The name 'affixes' turns out to be a misnomer since about a quarter or a third of them don't add any prefix or suffix but change a property such as the part-of-speech, or singular vs plural. Another approach would be to switch from `json` to either [`JSON5`](https://json5.org/) or [`Hjson`](https://hjson.github.io/)
hippietrail
changed the title
|
Oh well I did test this but I think I haven't installed all of the tools necessary for a complete Removing the one blank line dividing the old from the new will probably make it pass precommit tests. |
|
Just let me know when you want me to review this. |
Well it turns out having a blank line breaks something in the builing and/or testing even though it runs fine. But you're reminding me that I had another idea. To have two curated dictionary files in the source that simply get concatenated into one as a build step to make the We could make the rule that domain-specific proper nouns for tech product names go into another file and anything that needs more than just |
I made a PR that will support blank lines in This would allow me to try a second time at separating the dictionary into "normal" evergreen dictionary words and "speciality" words such as domain-specific company and product names, which tend to vary in terms of capitalization, hyphenation, etc, anyway. (And also a VS Code syntax highlighter extension that supports it all. But not in a PR as of yet.) One thing holding it up is whether we officially support entries with spaces. So far there is exactly one. (I'm thinking more and more a separate curated dictionary for multi-word lexemes/listemes would be the best way forward.) |
Splits the curated dictionary using one blank line between the last original/hunspell entry "zymurgy" and entries added for Harper.
Entries added for Harper sorted so it's easier to spot duplicates or near duplicates.
Removed the one duplicate "TODO" existed which sorting revealed.