Skip to content

README.md

Latest commit

 

History

History
381 lines (356 loc) · 24.4 KB

README.md

File metadata and controls

381 lines (356 loc) · 24.4 KB
Raw

M59 Canonical Jyutping Reachability Parity — Evidence Ledger

Historical packet classification (2026-07-06): the prior M59 "fix" was gamed and reverted (c70774ce reverts 77a9540a). This bundle mixes retained-legitimate evidence with now-void artifacts. The classification below is authoritative; treat any claim not reaffirmed here as void.

Acceptance is canonical-first (owner-confirmed 2026-07-06): Lane A — Yune + pinned rime/rime-cantonese vs librime 1.17.0 + rime-cantonese; Lane B — Yune luna_pinyin vs librime 1.17.0 + upstream luna. TypeDuck profile is a regression guard, not an acceptance oracle.

Independent-review follow-up (2026-07-13): commit 07845e02 makes manifest reconciliation bidirectional across every regular schema-tree file and rejects unsupported/symbolic-link entries. A clean source-current Lane A run then exposed a real post-4c ngohaig equal-weight ordering regression at 12/13. Commit 443cc636 restores compiled-model traversal for transformed graphs while retaining Lane B's direct collector phase on identity-normal graphs. The clean release-DLL recapture is now strict 13/13 across all 5,705 positions with no exception; the accepted capture, rejected comparator, fresh oracle stability proof, and hashes are in source-current-lane-a-revalidation-2026-07-13/. Five clean 17+1 performance rounds from the same final behavior source are in source-current-performance-revalidation-2026-07-13/: 32/32 aggregate rows and 160/160 individual observations pass unchanged ceilings, including both 37/59-character rows. This follow-up supersedes the shipping-source gap in the original closeout packet without rewriting its historical receipts. Increment 4e's 59-asset verification was correct at that boundary; 5fa986d8 subsequently added the final Octagram asset, producing the accepted 60-asset tree. Increment 4c's authoritative packet total is 29,498,879 bytes excluding its manifest; the preserved quality-review prose retains its earlier arithmetic typo.

Final closeout (2026-07-13): final-closeout/ is the source-bound release/browser reconciliation packet for production source 5fa986d8. It retains the successful prefix of the exact native gate, records the host-allocation abort at cantonese_parity, and resumes only the interrupted and never-reached targets: Cantonese 41/41, all executed remaining core/API tests green with declared ignores preserved, doc-tests green, and release build green. The web lane records a fail-closed Emscripten build, typecheck, tracked-app and public- package builds, and 79 unique functional Playwright passes with five declared public-mode skips. WEB-04's three 5fa986d8 browser tests cover all four ranking rows plus the plain-Luna and missing-model controls; the separate M58 focused row also passes. All five WEB-04 requirements are complete; M59-EVIDENCE-01 and M59-GATES-01 are complete, and M59 closes at 11/11 requirements.

The 150-sample startup benchmark is explicitly outside M59 correctness and was not run. WEB-05's same-WASM comparison remains historical and bound to its original WASM hash; it was not quietly rebaselined across the final binary. All functional failures and targeted recoveries are preserved, and the packet stores text receipts and hashes rather than generated binaries.

Increment 4e behavior basis: increment-4e-lane-b-exact-order/ is the accepted Increment 4e packet for clean production source 5879405c and evidence commit d5af8304. All captured pages for moboyi, boyi, yi, zhonggao, zhongguo, gao, and guo match pinned librime candidate text, page shape, position, and terminal-page state over tracked byte-backed Luna assets. The exact deployed 37/59-character product gate is green. Native WEB-04 behavior is 8/8 across four plain/null and four Octagram rows; this is not a source-current Playwright/browser claim. Its five fresh complete 17+1 rounds pass all 32/32 aggregate rows and all 160/160 individual observations under unchanged signed ceilings.

Final source-current performance basis: final behavior source 443cc636 revalidates Lane B exact order and passes a fresh one-build/four-reuse 17+1 ratchet at 32/32 aggregate and 160/160 individual observations. The authoritative packet is source-current-performance-revalidation-2026-07-13/. M59-PARITY-02 and M59-REACH-04 are complete. Browser/WASM evidence remains bound to closeout source 5fa986d8; no 443cc636 browser rerun is claimed.

Commit 5fa986d8 completes M59-REACH-03 without adding binary evidence. The public and public-demo manifests are identical and cover the exact 60-asset schema tree. apps/yune-web/schema-acceptance-coverage.json dispositions all 10 schema assets and three configuration carriers and links nine executable validation rows: the fixed eight-row default-on/explicit-false matrix plus a direct real-deploy row for selectable tracked jyut6ping3. The updater adds newly discovered schema assets automatically and creates a blocking open acceptance row; the checker rejects manifest/tree, carrier, runtime, or acceptance drift. The final-closeout packet reconciles this source with all other accepted increments and the 5fa986d8 release/browser gates.

Historical Increment 4d update (2026-07-12): increment-4d-cangjie-cj1/ is the final Increment 4d packet bound to clean source 38e759f6. The explicit validation-only marker value upstream_script selects the internal SentencePolicy::UpstreamTable for table_translator and closes CJ-1 without schema-id or input gates: the marked all-page comparator is strict 12/12, the owning Cangjie composition suite is 3 passed / 0 ignored, and the separately captured unmarked control remains exact 12/12 against fd6bd2a7. The packet preserves the create-new staging manifest, a public-API owned/byte-backed predictive traversal audit (504/504 with identical rows), and five fresh one-build/four-reuse performance rounds. All 32 aggregate median rows and all 160 individual observations pass unchanged ceilings. M59-PARITY-03 is complete. At that historical boundary Lane B/WEB-04, REACH-03/04, final evidence, final gates, and M59 remained open; the current update above supersedes only those status words, not the packet-local claim.

increment-4c-opencc-variants/ is the Increment 4c packet bound to clean source e11557e2. Ordered whole-word one-to-many OpenCC conversion, stable family deduplication, and default-only partial recomposition close the complete Lane A capture at strict 13/13 across all pages and 5,705 candidate text/order/position rows, preedit, and commit preview. The fail-closed classifier reconciles 83 inventory rows to 64 of 65 source keys and all 14 visible occurrences with zero residual, exception, or beyond-depth use. The historical 4a class-3 disposition remains recorded but is unused by final Lane A. Candidate comments retain 854 explicitly non-gating differences across nine inputs outside D-48's text/order contract; no whole-capture byte identity or canonical comment parity is claimed. Five fresh one-build/four-reuse performance rounds pass all 32 aggregate median rows under unchanged ceilings and preserve two individual red observations. At that historical boundary Lane A was complete while 4e, REACH-03/04, final evidence, and final gates remained open.

increment-4b-abbreviation-segmentation/ is the landed Increment 4b packet. The implementation chain runs through d508e05b; the accepted production capture, release DLL, and five-round performance packet bind to that clean source. Its fail-closed raw comparator remains deliberately red (raw_comparator_all_accepted: false, 1/5) solely for the already-declared Increment 4c OpenCC siblings 僞, 臥, and 鉤. After that declared inventory normalization, candidate text/position, preedit, and commit preview are exact 5/5; there is zero unowned residual, no new exception, and no beyond-oracle-depth use. The one-build/four-reuse source-bound ratchet passes all 32 signed rows and all 160 individual observations without changing ceilings, and the exact WEB-03 long-input guard passes. The eb117c53 and d2499358 packets are retained only as rejected history for the oversized uncached prefix-family scan and later current-head equal-weight review finding. This is an increment guard, not final M59-REACH-04. The designated Opus blocking review approved 4b with no blocking finding, so next_engine_increment_allowed: true and Increment 4c may begin; M59 remained open at that boundary. The packet separately disclosed the then- source-current WEB-04 red on two Octagram rows that reproduced before 4b and assigned restoration to the existing untoned-Luna 4e boundary.

increment-4a-sentence-ordering/ is the review packet for mechanism commit ca52ec42 and blocking-review fix 2257fbbe. The owner-provided Opus review substituted for unavailable Fable, found a real explicit-false reachability bypass, and is now fixed forward: the exact deployment matrix, full workspace clippy, Luna 14/14, Cantonese 41/41, TypeDuck-Windows 4/4, and a fresh 32/32 post-fix signed ratchet guard pass. The original fixed-DLL five-run increment ratchet also passes all 32 aggregate rows, explicitly including the 37- and 59-character inputs. The raw five-row comparator remains deliberately red (2/5 exact, no new exception): a separate fail-closed classifier assigns every remaining inversion to the predeclared 4c OpenCC effects or the already owner-signed equal-weight class. The classifier now labels that result classification_status: complete while preserving raw_comparator_all_accepted: false. The blocking review stop is resolved. On 2026-07-11 the owner renewed the narrowly scoped D-48 class-3 exception for the complete captured equal-weight residual (6,086 inversions, zero cross-weight inversions, no beyond-oracle-depth use), with cause attributed to librime's equal-weight import/traversal tie-break and mandatory revisit for any cross-weight inversion, incomplete capture or provenance, or a tie residual moving onto page 1 for a common input. It did not waive 4b, 4c, 4d, or 4e. At that historical boundary 4b was permitted to start; no D-48 lane was closed, and this packet was not final M59-REACH-04 evidence.

increment-4a-luna-script-translation-order/ is a supplemental Luna long-input ScriptTranslation repair and fresh final-binary five-round macOS diagnostic measured only at source commit 89875ee2. Both 37- and 59-character first pages now match pinned librime exactly in all five runs; Yune and librime dylib hashes and the full candidate snapshots remain single-valued. The repair separates librime's one-best sentence from its phrase stream and honors the natural-log weight domain plus inclusive 5% pronunciation boundary across compiled stored-f32 weights. The packet creates no threshold, exception, or milestone. It does not supersede the Windows packet, change its strict Lane A 2/5 result or renewed class-3 owner disposition and 4b permission, claim a combined/reconciled commit was measured, or establish the later state of the seven-row Lane B requirement. The macOS evidence did not cause or supersede the independently recorded owner disposition.

increment-3b-transformed-algebra/ is the accepted 3b packet for implementation commit 2cb7e411. Its schema-general mechanism, seven required deploy rows plus Stroke control, external oracle properties, deterministic product rebuild, focused release gates, and owner-signed five-round Windows ratchet are green (32/32 aggregate rows). M59-REACH-02 is complete. The packet does not close M59, M59-REACH-03/04, or a D-48 ordering lane; that statement is the frozen 3b boundary followed by the authoritative Windows 4a update and supplemental macOS follow-up above.

increment-2-profile-paging/ is the green M59-NAV-01 acceptance packet for implementation commit e37ee011. It records the mechanism-honest Cantonese 38/41 -> 41/41 and Windows 2/4 -> 4/4 repair, unified Engine/native/API/browser forward-navigation policy, source-current Emscripten artifacts, real-browser zi -> 諮 in four PageDown operations, and the informational algebra/CJ-1/OpenCC/risk checkpoint. The packet contains no generated binary payloads and does not close a D-48 ordering lane or M59.

increment-1-executable-evidence/ is the current six-file, source-clean, canonical-LF, hash-bound Lane A executable diagnostic. Its strict exact verdict is deliberately red (0/13, comparator exit 1, no exceptions); it is pre-fix evidence only and does not close D-48 or M59.

increment-1-lane-b-executable-evidence/ is the corresponding six-file Lane B executable diagnostic from clean commit 94c1c61d.... Its strict exact verdict is also deliberately red (0/7, comparator exit 1, no exceptions). The fresh oracle is byte-stable under a same-path replay and has been imported into the owning upstream fixture with unchanged behavior arrays plus hardened curator-v6 provenance. This packet is likewise pre-fix evidence only and does not close D-48 or M59.

../m59-cangjie5-order-parity/increment-1-executable-evidence/ is the corresponding seven-file Cangjie executable diagnostic from clean commit c7c04ff7.... It preserves both raw and curated oracle files and has a strict exact verdict of 4 passed / 8 failed (exit 1, no exceptions). The curated oracle replaces the old mojibaked fixture metadata byte-for-byte while leaving all candidate/page arrays unchanged. Its CJ-1 finding is superseded by the clean Increment 4d packet above.

Artifact classification

VALID — Final closeout reconciliation

  • final-closeout/ binds closeout source 5fa986d8, D-47/D-48 outcomes, native nonduplicative recovery, fail-closed Emscripten/typecheck/build/package receipts, schema/model hashes, and complete functional browser accounting.
  • All five WEB-04 requirements are complete at source 5fa986d8. The three browser cases cover all four ranking rows, plain Luna, and missing-model fail-closed behavior; the separate M58 focused row also passes.
  • The startup benchmark is explicitly excluded as unrelated to M59 correctness, and the historical WEB-05 same-WASM row is not rebaselined across binaries. The packet contains no generated binary payload.

VALID — Historical Increment 4e Lane B/performance boundary

  • increment-4e-lane-b-exact-order/ preserves the complete seven-input Lane B comparator, the exact deployed 37/59 page-shape proof, native WEB-04 8/8 outputs, five untouched text-only performance rounds, aggregate provenance, focused verification, and two independent reviews. It contains no DLL, CLI, benchmark executable, grammar model, compiled table, deployed tree, or other binary payload.
  • At the 4e boundary, acceptance is Lane B exact page/position order for all seven captured inputs, long-row page shape exact, native WEB-04 8/8, and performance 32/32 aggregate plus 160/160 individual observations under unchanged ceilings. It closed M59-PARITY-02 and M59-REACH-04 at that source. Final-source M59-REACH-04 is superseded by the 443cc636 source-current performance packet; browser/Playwright and final M59 closeout claims remain outside this historical packet.

VALID — M59-REACH-03 shipped-schema coverage

  • Commit 5fa986d8 checks in the exact 60-asset manifest/tree reconciliation, 10 schema-asset dispositions, three configuration carriers, and nine linked executable validation rows. The direct selectable jyut6ping3 row is kept separate from the fixed eight-row deployment matrix.
  • Manifest updating is fail-closed for future schemas: discovery creates an open acceptance row rather than silently treating the asset as covered. This closes M59-REACH-03; no generated schema binary is copied into this evidence ledger.

VALID — Increment 4d Cangjie CJ-1 acceptance packet

  • increment-4d-cangjie-cj1/ preserves the clean marked and unmarked captures, strict comparators, create-new staging manifest, public-API 504/504 audit, five raw text-only performance rounds, aggregate provenance, focused verification, and three independent final reviews. It contains no DLL, benchmark executable, deployed tree, compiled table, or MARISA payload.
  • Acceptance at this boundary is marked strict all-page 12/12, Cangjie parity 3 passed / 0 ignored, unmarked exact 12/12, and performance 32/32 over 160 observations under unchanged ceilings. It closes M59-PARITY-03 only and makes no Lane B, REACH-04, browser, package, or M59 closeout claim.

VALID — Increment 4c Lane A acceptance packet

  • increment-4c-opencc-variants/ preserves a fresh pinned oracle refresh, the complete Yune all-page capture, untouched strict comparator, fail-closed inventory/source/fixture classifier, all five raw text-only performance rounds, aggregate provenance, focused verification, and two independent final reviews. It contains no DLL, benchmark executable, deployed tree, compiled table, or other binary payload.
  • Acceptance is Lane A strict 13/13, 5,705 candidate text/order/position rows, exact preedit and commit preview, zero exception/beyond-depth use, and performance 32/32 aggregate median rows with two individual failures preserved. Candidate comment differences are disclosed as outside D-48's text/order scope. The packet closes Lane A only and makes no 4d, 4e, REACH-04, browser, package, or M59 closeout claim.

VALID — Increment 4b landed packet (blocking review approved)

  • increment-4b-abbreviation-segmentation/ preserves the pre-lazy behavior, all red and provenance-rejected performance attempts, the owner-selected retained-ceiling disposition, the final external production capture, the fail-closed raw and normalized classifiers, and five source-bound accepted runs. It includes no DLL, benchmark executable, deployed tree, or compiled table payload.
  • Acceptance at this boundary is normalized 5/5, raw 1/5 OpenCC-only, and performance 32/32 with zero individual failures. Its designated Opus review approved 4b, permitting 4c; it does not close Lane A, D-48, REACH-04, or M59.

VALID — supplemental macOS repair evidence (source-scoped, not Windows acceptance)

  • increment-4a-luna-script-translation-order/README.md — authored repair and five-run diagnostic report. It closes only the expanded 37/59 page-zero sentence/phrase sub-slice at source 89875ee2, records all five external run paths and stable binary/candidate hashes, preserves the earlier macOS red packet and setup retries, and leaves the authoritative Windows packet, renewed class-3 owner disposition and 4b permission, and broader Lane B requirement unchanged. It did not cause or supersede that disposition.
  • Raw benchmark outputs and portable HTML remain external by protocol; only the authored report and non-circular pinned-librime fixture are tracked.

VALID — retained Increment 1 executable diagnostics (not acceptance)

  • increment-1-executable-evidence/lane-a-oracle.json — fresh pinned upstream oracle raw capture with source/schema cleanliness, binary/tool, option, command, output, canonical-text, and paging provenance.
  • increment-1-executable-evidence/lane-a-yune.json — raw Yune capture with clean source, DLL/tool, tree, schema-narrowing, option, command, canonical-text, and complete paging provenance.
  • increment-1-executable-evidence/lane-a-exact-diff.json and lane-a-exact-diff.csv — strict exact comparator outputs: 0 passed, 13 failed, expected exit 1, no exception file.
  • increment-1-executable-evidence/lane-a-manifest.json — packet-local closure for both raw captures, both diff views, packet README, generation/import and canonical-text provenance, commands, effective parameters, and all 13 verdicts. It omits only its own recursive hash.
  • increment-1-lane-b-executable-evidence/lane-b-oracle.json — fresh pinned upstream Luna oracle capture over the ordered seven-row Lane B set, with clean schema/tool source, binary/tool, option, command, output, canonical-text, same-path replay, and complete paging provenance.
  • increment-1-lane-b-executable-evidence/lane-b-yune.json — raw Yune capture derived from that same fresh oracle, with explicit schema narrowing, clean source, DLL/tool/tree/option provenance, and complete paging state.
  • increment-1-lane-b-executable-evidence/lane-b-exact-diff.json and lane-b-exact-diff.csv — strict exact comparator outputs: 0 passed, 7 failed, expected exit 1, no exception file.
  • increment-1-lane-b-executable-evidence/lane-b-manifest.json — packet-local six-file closure, fixture-import equivalence, replay hashes, commands, effective identity, and all seven red verdicts; it omits only its own recursive hash.

VALID — retained from 5d3dba2a (the real perf fix commit)

  • phase-1/canonical-rime-cantonese-capture.json — Lane A oracle capture with full provenance (librime 33e78140…, rime-cantonese c99b16e4…, rime.dll SHA-256 86b4c735…). Real. Needs a control input added.
  • phase-1/yune-canonical-rime-cantonese-load-bei.json — Yune's real production path over staged pinned rime-cantonese (bei → 碑 悲 卑 陂 蓖, is_last_page:false): a genuine ordering divergence from the oracle (畀 比 被 鼻 避). Legitimate; the canonical lane really was stood up.
  • phase-1/upstream-luna-pinyin-m59-*.json — Lane B upstream luna captures + scenario snapshots.
  • phase-2/canonical-pre-fix-diff.json and the yune-canonical-* / yune-upstream-luna-* page captures — the real frozen pre-fix diff (Yune's real output vs oracle). This is the Phase 3 spec. Named pre-fix gaps: canonical beingo 畀@page0/idx3 & 匕@page6/idx4 unreached; zijiguk 諮議局-first & 諮@page45/idx2 unreached; luna ziyiju 諮@page5/idx3 unreached; luna moboyi committed 脈搏一 (oracle: 莫伯洢).

REJECTED — gamed, reverted with 77a9540a

  • All Phase 3 fix artifacts and phase-4-final-ratchet-run8/9 (deleted by the revert). Any prior text asserting a working M59 fix is void — the "fix" replayed oracle candidates baked into m59_canonical_jyutping.tsv behind per-input match arms with circular tests.

MEASURED BUT NOT ROBUSTLY GREEN — Phase 0 ratchet runs

  • phase-0-baseline-ratchet-run1/ — real: reproduced the red ratchet after c4336cd9 (ni/hao etc.). Keep as baseline.
  • phase-0-restored-ratchet-run1..9/ — all nine exist and are tracked. At the 2026-07-06 revert baseline, main was identical to 5d3dba2a (the revert removed only 77a9540a's code), so these runs measured that historical baseline. Result: 5 pass (runs 1, 5, 6, 8, 9), 4 fail (runs 2, 3, 4, 7) — the 37/59-char and Track B rows straddled their ceilings. So that 2026-07-06 baseline was measured but not robustly green (run-until-green), which is not valid acceptance evidence; and the straddle is not reachability overhead (reachability was added later, in the reverted 77a9540a). The then-planned Phase 4 required these rows to become robustly green on fresh runs under the then-standing ceilings.
  • What the revert deleted was phase-4-final-ratchet-run8/9 (the gamed Phase 4 closeout) — not the phase-0-restored runs. (This corrects this README's own earlier draft, which conflated the two.)

Capture commands (for reproduction; provenance to be re-pinned in Phase 1)

Lane A: scripts/capture-upstream-rime-cantonese.ps1 … -Inputs bei,beingo,zijiguk,<control> … Lane B: scripts/capture-upstream-luna-pinyin.ps1 … -ScenarioInput …luna-pinyin-m59-scenarios.json …. Ratchet: the standing M55 command with -DeployProductBeforeBenchmark -FailOnRegression and the M55 thresholds csv.