Ukrainian language features - #2478
Open
JayXT wants to merge 3 commits into
Open
Conversation
Ukrainian was registered in `language-descriptors.js` with capitalization
preprocessors only. This adds the advanced language features described in
`docs/development/language-features.md`.
Text preprocessors (`uk/ukrainian-text-preprocessors.js`):
- `removeUkrainianDiacritics` removes the combining acute accent used to
mark stress. Ukrainian dictionaries are inconsistent about including it
and running text from textbooks and reference works frequently does.
- `ukrainianApostropheVariants` treats the several characters used for the
Ukrainian apostrophe (U+0027, U+2018, U+2019, U+02BC, U+0060, U+00B4) as
equivalent, so that scanning "п’ять" finds a "п'ять" entry.
Language transforms (`uk/ukrainian-transforms.js`) cover:
- Noun declension for all four declensions, in seven cases and two numbers,
including the о/е to і alternation in closed syllables ("стола" -> "стіл"),
the zero-ending genitive plural ("книг" -> "книга") and the lengthened
consonant of the third-declension instrumental ("сіллю" -> "сіль").
- Hard- and soft-stem adjective declension, comparatives and superlatives.
- Verb conjugation for both conjugations: present/simple future, synthetic
future, past, imperative, participles and verbal nouns, with the usual
first-person singular consonant mutations ("ходжу" -> "ходити"). Every
verb ending is also generated in its reflexive shapes, whose dictionary
form keeps the postfix ("вчуся" -> "вчитися").
The condition names (`n`, `adj`, `adv`, `v`) match the part-of-speech codes
that wiktionary-to-yomitan writes into the term bank rules field for other
languages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VYKqWh9jL3GCzTzZZA5LVn
Measured against the 127 272 form/lemma pairs recoverable from the
`non-lemma` glosses in wty-uk-en, the initial rules recovered the correct
lemma for 71.3% of inflected forms. Each cluster below was added only after
confirming it earns more than it costs; together they take that to 78.9%
while raising the number of unique deinflection candidates per word from
5.78 to 6.06.
- Hard-stem adjectives have an alternative locative singular in -ім beside
-ому ("гарнім" -> "гарний"). Only the soft-stem variant was handled, so
this class scored 0%.
- Imperatives of the productive -увати/-ювати verb class
("маринуймо" -> "маринувати").
- Masculine nouns in -ок and -ець/-єць drop the vowel before an ending
("підвечірками" -> "підвечірок", "латвійцем" -> "латвієць").
- The plain masculine locative singular in -і; previously only the variant
with the о/е to і alternation was covered ("харків'янині" -> "харків'янин").
- The colloquial short first-person plural in -м beside the standard -мо
("ухвалюєм", "ухвалюватимем").
- Soft-stem feminine genitive plural ("сесій" -> "сесія"), and the third
declension genitive and vocative reached through the existing alternation
helper ("щільности", "радосте" -> "радість").
Per-cluster contribution to recall, measured by ablation: short -м +1.66,
adjective -ім +1.61, -увати imperatives +1.38, locative -і +1.10,
-ець/-єць +0.74, -ок +0.62, genitive -ій and -ости +0.28, vocative +0.15.
Deliberately excluded: extending the short first-person plural to the bare
-ем/-им endings, which adds only +0.44 recall for +0.08 candidates per word
because it loads two of the most common noun and adjective endings; and the
derivational relations in the gloss data (aspect pairs, relational
adjectives, abstract nouns, spelling variants), which a suffix table cannot
express without heavy over-generation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VYKqWh9jL3GCzTzZZA5LVn
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 282704acc8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
The apostrophe processor recognised six characters on input but emitted
only U+0027 and U+2019, which made the mapping one-way. Dictionary terms
are indexed exactly as the dictionary spells them and no normalization is
applied at import, so an entry stored with U+02BC was unreachable from any
other spelling. wty-uk-en contains such an entry ("розвʼязання"), and
U+02BC is the character Unicode designates for the Ukrainian apostrophe,
so other dictionaries may well use it throughout.
A variant is now produced for each of the three characters that occur as
headword spellings in practice. Words containing no apostrophe are
unaffected in cost, since the translator collects variants in a Map keyed
by the resulting string and identical outputs collapse.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VYKqWh9jL3GCzTzZZA5LVn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ukrainian was registered in
language-descriptors.jswith capitalization preprocessors only. This adds the advanced language features described indocs/development/language-features.md.Text preprocessors (
uk/ukrainian-text-preprocessors.js):removeUkrainianDiacriticsremoves the combining acute accent used to mark stress. Ukrainian dictionaries are inconsistent about including it and running text from textbooks and reference works frequently does.ukrainianApostropheVariantstreats the several characters used for the Ukrainian apostrophe (U+0027, U+2018, U+2019, U+02BC, U+0060, U+00B4) as equivalent, so that scanning "п’ять" finds a "п'ять" entry.Language transforms (
uk/ukrainian-transforms.js) cover:The condition names (
n,adj,adv,v) match the part-of-speech codes that wiktionary-to-yomitan writes into the term bank rules field for other languages.