Skip to content

bho: revive the manifest-header gate, and file the अ/ʌ split - #1029

Merged
christopherthompson81 merged 3 commits into
mainfrom
bho/review-followup
Aug 26, 2026
Merged

bho: revive the manifest-header gate, and file the अ/ʌ split#1029
christopherthompson81 merged 3 commits into
mainfrom
bho/review-followup

Conversation

@christopherthompson81

Copy link
Copy Markdown
Owner

Follow-up to #1025, from its own review.

The manifest-header gate has been dead since the data move

tools/check-manifest-headers.mjs globs src/languages/**/*.jsonc. All 184 manifests live under data/languages/. The glob matches nothing, so every run takes the script's own "refuses to pass vacuously" branch and exits 2.

Loud, so nobody was ever misled by a false pass — but the guard against a header detached from the key it describes (the defect it was written for, four times over: #746, #750, #751, #755) has had zero coverage since #769 moved the data tree out of src/. That matters for #1025 specifically, which was largely a rewrite of four top-level key headers in a manifest.

Reviving it exposed a second defect in the same script

It immediately fired on #1025's independentVowels header — a caveat appended below the original line, still directly above its own key:

✗ data/languages/bhojpuri/bhojpuri.jsonc: "independentVowels" no longer carries its own header
   was: // Independent vowel letters (word-initial / standalone).
   now: // Independent vowel letters (word-initial / standalone).

The script already exempts a header replaced outright — its comment says flagging those "would train people to never improve a stale comment" — but that exemption keys on the old text being gone from the file entirely. An extension leaves the old text in the file and above its key, so it is indistinguishable from an orphaning by the normalized.includes test alone. That is the same "train people to ignore the gate" failure from the other direction.

In a real orphaning the old text has moved to a different key, so now cannot contain it. That is the discriminator, and it is now the guard. Verified both ways:

  • passes the extended header → ok — 1 changed manifest(s), every pre-existing key kept its own header
  • still catches a synthetic key inserted between vowelSigns' header and its table → ✗ … "vowelSigns" no longer carries its own header / now: (none — a new block was inserted between them)

Found, not fixed — one phoneme, two symbols

⟨अ⟩ ships [ʌ] while inherentVowel ships [ə], and the latter's own comment calls the [ʌ] realisation the same sound, "folded". So अञ्जन reads ʌnd͡ʒən — both symbols, one word, two syllables apart.

provenance compounds it: the declared inventory is /i e ɛ a ʌ ɔ o u/, which contains ʌ and not ə. Neither value matches the file's own statement of the system.

Not cosmetic — ʌ is in 99 of the 200 golden rows, ə in all 200.

Every non-derived Devanagari sibling writes ə here (hi/mr/awa/mai/hne); only magahi.jsonc, derived from this file, copies the ʌ. rangpuri.jsonc also splits (अ→ɔ over an ə inherent) but documents an Eastern-Indic reason; this file documents none.

Filed rather than fixed because the direction is genuinely unobvious: folding ʌ→ə matches the inherent vowel and the siblings, folding ə→ʌ matches the declared inventory. The 1622-pair grammar-mined referee decides it, and it is not in the repo.

Also recorded: provenance ends "Single published source → ." — a generator dropped the confidence grade after the arrow, and magahi.jsonc carries the same wound. Marked as truncated rather than reconstructed; guessing a confidence grade is worse than showing that one is missing. (This is a shipped string, not a comment — #1025 hunted exactly this shape in the symbols comment and left the one in data.)

Not done

csharp/STATUS.md is deliberately untouched: four ports are in flight concurrently and the porting brief reserves that file to avoid four-way conflicts. It currently under-reports the gate (says 69 codes / 13,800 rows; actual is 79 / 15,800) and should be reconciled once the batch lands.

Gates

Comment- and string-only. bho 200/200; full gate 79 languages, 15,800 rows, 0 differ; 5,397 vitest passed (5 skipped); 693 dotnet passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WuH4pRrB1PAEiG8CCLVR5S

christopherthompson81 and others added 2 commits August 25, 2026 22:55
Follow-up to #1025, from its own review.

⚠ tools/check-manifest-headers.mjs HAS BEEN DEAD SINCE THE DATA MOVE. It globs
`src/languages/**/*.jsonc`; all 184 manifests live under `data/languages/`.
The glob matches nothing, so every run takes the "refuses to pass vacuously"
branch and exits 2 — loud, so nobody was misled by a false pass, but the guard
against a header detached from its key has had zero coverage since #769. That
matters here specifically: #1025 was largely a rewrite of four top-level key
headers in a manifest.

Reviving it exposed a second defect in the same script. It fired on #1025's
extended `independentVowels` header — a caveat APPENDED below the original
line, still directly above its own key. The script's existing exemption only
covers a header replaced outright ("old text gone from the file entirely"), so
an extension looks exactly like an orphaning to the `normalized.includes` test.
In a real orphaning the old text has moved to a DIFFERENT key, so `now` cannot
contain it — that is the discriminator, and it is now the guard. Verified both
ways: the gate passes the extended header, and still catches a synthetic
insertion between `vowelSigns`' header and its table.

FILED, NOT FIXED — ONE PHONEME, TWO SYMBOLS. ⟨अ⟩ ships [ʌ] while
`inherentVowel` ships [ə], and the latter's own comment calls [ʌ] the same
sound, "folded". अञ्जन reads ʌnd͡ʒən: both symbols, one word. `provenance`
compounds it — the declared inventory /i e ɛ a ʌ ɔ o u/ contains ʌ and not ə,
so neither value matches the file's own statement of the system. Not cosmetic:
ʌ is in 99 of the 200 golden rows, ə in all 200. Every non-derived Devanagari
sibling writes ə (hi/mr/awa/mai/hne); only magahi.jsonc, derived from this
file, copies the ʌ. The direction is genuinely unobvious — folding ʌ→ə matches
the inherent vowel and the siblings, folding ə→ʌ matches the declared
inventory — and the referee that decides it is not in the repo.

Also recorded: `provenance` ends "Single published source → ." — a generator
dropped the confidence grade after the arrow, and magahi.jsonc has the same
wound. Marked as truncated rather than reconstructed; guessing a confidence
grade is worse than showing one is missing.

csharp/STATUS.md is deliberately untouched — four ports are in flight and the
brief reserves that file.

Comment- and string-only: bho 200/200, full gate 79 languages / 15,800 rows /
0 differ, 5,397 vitest, 693 dotnet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuH4pRrB1PAEiG8CCLVR5S
Running the gate from the previous commit over a wider window turned up the
exact defect it was written for, sitting in main.

amharic.jsonc: the four-line "THE SHARED SYMBOL TIER's data" header ended up
above the ORDINALS block. `ordinals` and `words` were inserted between that
header and the `symbolTier` key it describes, so a reader following the header
reached the ordinals table and `symbolTier` — 60 lines further down — carried
no header at all. This is #746/#750/#751/#755 verbatim. The header is moved
back onto its key; nothing else changed, and am stays 200/200.

Two further hardenings the wide run exposed, both of the script's own stated
"refuses to pass vacuously" invariant:

- `git show` wrote "exists on disk, but not in <base>" to the terminal for
  every file when the base predates the data-tree move. Piped, so it reaches
  the catch instead of the console.
- That catch means "no version at base", which is usually a new manifest but is
  EVERY file across a tree move — and the script then printed "ok" having
  compared nothing, which is precisely the vacuous pass its docstring says let
  #755 ship. Comparisons are now counted: an "ok" states how many files it
  actually compared, and zero-compared exits 2 like the empty-diff branch.

Verified all three ways: passes clean on a normal base (35 changed manifests,
35 compared), exits 2 on a pre-move base, and still flags a synthetic key
inserted between vowelSigns' header and its table.

Full gate 79 languages / 15,800 rows / 0 differ.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuH4pRrB1PAEiG8CCLVR5S
@christopherthompson81

Copy link
Copy Markdown
Owner Author

Pushed a second commit. Running the revived gate over a wider window (origin/main~30) turned up the exact defect it was written for, sitting in main:

✗ data/languages/amharic/amharic.jsonc: "symbolTier" no longer carries its own header
   was: // ------------------------------------------------------------------------------
   now: (none — a new block was inserted between them)

ordinals and words had been inserted between the four-line "THE SHARED SYMBOL TIER's data" header and the symbolTier key it describes. A reader following that header reached the ordinals table, and symbolTier — 60 lines further down — carried no header at all. #746/#750/#751/#755 verbatim, found on the gate's first wide run after eight months dead. Header moved back onto its key; am stays 200/200.

Two further hardenings the same run exposed, both of the script's own "refuses to pass vacuously" invariant:

  • git show was writing exists on disk, but not in <base> to the terminal for all 184 files when the base predates the data-tree move. Now piped into the catch.
  • That catch means "no version at base" — usually a new manifest, but every file across a tree move. The script then printed ok having compared nothing, which is exactly the vacuous pass its docstring says let latin: the three function-local phone classes move to the manifest #755 ship. Comparisons are now counted: an ok states how many files it actually compared, and zero-compared exits 2 like the empty-diff branch.

Verified all three ways:

base result
origin/main~30 (normal) ok — 35 changed manifest(s), 35 compared, exit 0
origin/main~150 (pre-move) NOTHING WAS ACTUALLY COMPARED, exit 2
synthetic key inserted before vowelSigns' table ✗ … (none — a new block was inserted between them), exit 1

I deliberately kept the new-manifest case a pass rather than a failure — a PR that only adds a new language legitimately compares nothing, and failing it would train people to ignore the gate, the same trap the existing "header edited in place" exemption guards against. It reports the count instead.

Self-review of the previous commit: the zero-compared guard ran AFTER the
console.log, so the pre-move-base run printed "ok — 184 changed manifest(s), 0
compared…" and only then errored. A gate that says ok on its way to exit 2 is
worse than one that just says ok. Moved above the log, and the message now
names the actual cause and the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuH4pRrB1PAEiG8CCLVR5S
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant