Skip to content

core/ipa: ⟨ɑ ɒ ʏ⟩ join the nucleus class — stress was vanishing, bhojpuri +125 words - #752

Merged
christopherthompson81 merged 2 commits into
mainfrom
ipa-alpha-nucleus
Aug 8, 2026
Merged

core/ipa: ⟨ɑ ɒ ʏ⟩ join the nucleus class — stress was vanishing, bhojpuri +125 words#752
christopherthompson81 merged 2 commits into
mainfrom
ipa-alpha-nucleus

Conversation

@christopherthompson81

@christopherthompson81 christopherthompson81 commented Aug 8, 2026

Copy link
Copy Markdown
Owner

The last residual from #748 — the one I twice said was blocked. It wasn't, and both the diagnosis and the evidence turned out to be available all along.

What I kept missing

I deferred this on the grounds that the ur/bho referees carry no stress marks, so stress PLACEMENT can't be adjudicated. True — but placement was never the only thing at stake. I never asked what happens to a word with no visible nucleus at all:

applyWeightStress("kaːm")    → kˈaːm        applyWeightStress("kɑːm")    → kɑːm
applyWeightStress("saːlaːɾ") → saːlˈaːɾ     applyWeightStress("sɑːlɑːɾ") → sɑːlɑːɾ

The weight rule's fallback chain ends at "first syllable" — but with no nucleus visible there are no syllables, so it emits nothing. In the live engine: آپ "you" → ɑːp, کام "work" → kɑːm, بازارbɑːzɑːɾ. Everyday words, silently unstressed. No referee is needed to call that wrong.

And the referee did adjudicate — through the other language

referee main with ⟨ɑ ɒ ʏ⟩
ur.wikipron-urd-broad 4382/7709 (56.8%) 4382/7709 — identical
ur.cle-speech 3364/5667 (59.4%) 3364/5667 — identical
bho.grammar-mined 1008/1623 (62.1%) 1133/1623 (69.8%)

+125 words. Urdu really is stress-folded and can't see the change; Bhojpuri can, because IPA_VOWELS is not only the stress nucleus — the Indic engines build SEGMENTAL regexes from it too, so an invisible ⟨ɑ⟩ was corrupting more than stress. The evidence I said didn't exist was one command away, in the corpus I'd already been told was in-repo.

The golden changes

18 updated, and they split:

  • 8 are stress appearing where there was noneɑːbɑːd̪→ɑːbˈɑːd̪, ɦɑːt̪ʰ→ɦˈɑːt̪ʰ, ɑːʋɑːz→ɑːʋˈɑːz, nɑːm→nˈɑːm, ɦzɑːɾ→ɦzˈɑːɾ, mʋɑːd̪→mʋˈɑːd̪, sɑːʈʰʋɑː̃→sˈɑːʈʰʋɑː̃, mʊɾəbbˈɑːʔ.
  • The rest are placement moving to what the documented rule (rightmost superheavy → rightmost non-final heavy → first) gives once it can see every syllable. Not referee-confirmed for Urdu; the Bhojpuri number is the evidence that the corrected parse is better overall.

A new test pins the bare-word case directly, so the actual bug can't come back quietly.

Also

IPA_VOWELS and IPA_VOWEL collapse back into one list. The split #748 introduced was a workaround for a question that had an answer.

npm run ci green — 228 files / 3137 tests. docs/ipa_classes_investigation.md Run 9 records the miss: I scoped the question to the first thing I noticed and treated one corpus's silence as the whole evidence base.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr


Review addendum

Independent measurement confirmed the blast radius: gu/mr/pa/hi change 0 words, sd changes 18 (stress-only), ur changes 2550 — all stress-only, zero segmental, so the claim that the Urdu referees can't see it is exact. Every referee score is identical except bho.

The bho +125 breaks down as 135 fixed / 10 broken, and the 10 are one class: words written with the avagraha ⟨ऽ⟩ (U+0973), which appears in neither the Bhojpuri nor Hindi engine and is dropped — the invisible ⟨ɑ⟩ had been accidentally suppressing their final-schwa deletion. Removing the accident exposed a pre-existing gap rather than creating one (avagraha matches 15/31 → 5/31). Scope is 31 bho + 2 mai words; nothing else in any referee carries the character. Recorded in the investigation doc and left as its own change, since schwaDeletion in the manifest is already the hook for it.

Also confirmed: no residual "no stress at all" outputs remain except genuinely vowelless tokens (ء→ʔ, ળ→ɭ), and IPA_VOWELS is now the complete IPA vowel chart with no regex metacharacters, so [${IPA_VOWELS}] stays safe to interpolate.

christopherthompson81 and others added 2 commits August 7, 2026 23:22
…bhojpuri gains 125 words

I deferred this twice on the grounds that the referees carry no stress marks, so placement could not be
adjudicated. Both times I asked only which placement was right, and never asked what happens to a word
with NO visible nucleus. The answer is that weightStress emits nothing at all: آپ 'you' → ɑːp, کام
'work' → kɑːm, بازار → bɑːzɑːɾ, all bare, while the minimal pair kaːm → kˈaːm was stressed all along.

No referee is needed to call that wrong. And the referee then adjudicated anyway, through the other
language: ur is unchanged on both its corpora (4382/7709 and 3364/5667 — genuinely stress-folded), while
bho goes 1008 → 1133/1623, 62.1% → 69.8%. IPA_VOWELS is not only the stress nucleus; the Indic engines
build segmental regexes from it too, so an invisible ⟨ɑ⟩ was corrupting more than stress.

18 goldens updated. Eight are stress APPEARING where there was none (آباد ہاتھ آواز نام ہزار مواد
مربع 60واں); the rest are placement moving to what the documented weight rule gives once it can see
every syllable. A test now pins the bare-word case directly.

IPA_VOWELS and IPA_VOWEL become one list again — the split in #748 was a workaround for a question that
had an answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr
The bhojpuri gain is 135 fixed / 10 broken, and the 10 are one class: words written with ⟨ऽ⟩ (U+0973),
which appears in neither engine and is dropped, while the invisible ⟨ɑ⟩ had been accidentally
suppressing their final-schwa deletion. Removing the accident exposed a real gap rather than creating
one. Logged with its scope (31 bho + 2 mai words, nothing else in any referee) as its own change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr
@christopherthompson81
christopherthompson81 merged commit 12729bb into main Aug 8, 2026
2 checks passed
@christopherthompson81
christopherthompson81 deleted the ipa-alpha-nucleus branch August 8, 2026 05:29
christopherthompson81 added a commit that referenced this pull request Aug 8, 2026
…1133 → 1153

The residual under #752's +125: ⟨ऽ⟩ (U+093D — not U+0973 as reported; I dumped the codepoints) is
unknown to the engine and dropped, and the invisible ⟨ɑ⟩ had been accidentally suppressing final-schwa
deletion for these forms. Removing that accident exposed the real gap.

In Bhojpuri the avagraha is a LIVE orthographic mark on the imperative/participial forms, not Sanskrit's
elision sign: it writes exactly the inherent vowel the schwa rule removes, so कर/करऽ is a minimal pair.
All 31 forms in the grammar-mined referee keep the vowel and none delete it.

A retainOnAvagraha flag on the manifest's schwaDeletion block, tested against the SPELLING — g2p drops
the character, so nothing survives into the phones to condition on. That makes it the only
retain-condition not decidable from the IPA string, which the comment now says.

bho 1133 → 1153/1623 (69.8% → 71.0%), mai 141 → 142/167. Opt-in, so the other 15 engines composing
makeNativeHindi are untouched — hi/mr/ne/awa re-measured identical to main.

Deliberate residual: Maithili's referee transcribes the retained vowel LONG and we emit short. Retention
is right in both; the length is two words' worth of Maithili detail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr
christopherthompson81 added a commit that referenced this pull request Aug 8, 2026
…1133 → 1153 (#754)

* bhojpuri/maithili: a word-final avagraha ⟨ऽ⟩ retains the vowel — bho 1133 → 1153

The residual under #752's +125: ⟨ऽ⟩ (U+093D — not U+0973 as reported; I dumped the codepoints) is
unknown to the engine and dropped, and the invisible ⟨ɑ⟩ had been accidentally suppressing final-schwa
deletion for these forms. Removing that accident exposed the real gap.

In Bhojpuri the avagraha is a LIVE orthographic mark on the imperative/participial forms, not Sanskrit's
elision sign: it writes exactly the inherent vowel the schwa rule removes, so कर/करऽ is a minimal pair.
All 31 forms in the grammar-mined referee keep the vowel and none delete it.

A retainOnAvagraha flag on the manifest's schwaDeletion block, tested against the SPELLING — g2p drops
the character, so nothing survives into the phones to condition on. That makes it the only
retain-condition not decidable from the IPA string, which the comment now says.

bho 1133 → 1153/1623 (69.8% → 71.0%), mai 141 → 142/167. Opt-in, so the other 15 engines composing
makeNativeHindi are untouched — hi/mr/ne/awa re-measured identical to main.

Deliberate residual: Maithili's referee transcribes the retained vowel LONG and we emit short. Retention
is right in both; the length is two words' worth of Maithili detail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr

* review: the avagraha sign is per-SCRIPT, and maithili had no test behind its flag

Three fixes, the middle one substantive:

- /\u093D$/ hard-coded the DEVANAGARI avagraha while makeNativeHindi is script-parameterized and
  gujarati already passes its own block. Any non-Devanagari language setting the flag would have got a
  manifest that reads as enabled while every final schwa still deleted. The sign now lives on
  AbugidaScript (Brahmic blocks are aligned — offset 0x3D in each: ऽ U+093D, ঽ U+09BD, ઽ U+0ABD), and a
  manifest asking for retention under a script that declares none THROWS at construction instead of
  doing nothing quietly. Verified the throw fires rather than assuming it.

- maithili opted in with no rationale and nothing asserting it; a later re-tune of its schwa block could
  have flipped it back silently. It now carries the reason, the thin evidence (two referee forms), and
  goldens — including the known length divergence, pinned to what the engine does rather than what the
  referee wants.

- the Run 10 heading was dated ahead of the commit it records.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Lc3WnUgogC7okV7n53vjr

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant