Skip to content

Return a most-likely path from forced_align when predecessors tie - #4222

Open
happyarts wants to merge 1 commit into
pytorch:mainfrom
happyarts:forced-align-tie
Open

Return a most-likely path from forced_align when predecessors tie#4222
happyarts wants to merge 1 commit into
pytorch:mainfrom
happyarts:forced-align-tie

Conversation

@happyarts

Copy link
Copy Markdown

Fixes #4221

forced_align can return an alignment that is not a most-likely path. The per-cell choice is

if (x2 > x1 && x2 > x0)      { result = x2; backPtr = 2; }
else if (x1 > x0 && x1 > x2) { result = x1; backPtr = 1; }
else                         { result = x0; backPtr = 0; }

and when x1 == x2 with both above x0, neither condition holds, so the else branch takes x0 — the strictly worst of the three. Both the CPU and the CUDA kernel carry the identical chain, so both are corrected here.

Why x1 > x0 alone is correct

Reaching the second branch already implies x2 is not strictly greater than both others:

  • branch 1 returns x2 with x2 > x1 and x2 > x0 — the maximum;
  • branch 2 returns x1 with x1 > x0; had x2 > x1 held, then x2 > x1 > x0 would have taken branch 1, so x2 ≤ x1 — the maximum;
  • branch 3 returns x0 with x0 ≥ x1; had x2 > x0 held, then x2 > x0 ≥ x1 would have taken branch 1, so x2 ≤ x0 — the maximum.

So the only behaviour that changes is that exact ties now route to the larger value instead of to x0.

Verification

  • Exhaustive, against the best score over all enumerated valid paths, on 3243 random small cases (T ≤ 5, vocabulary ≤ 4, integer log-probabilities so ties are exact): 92 sub-optimal before, 0 after, worst deficit 3.0 log-probability.
  • No change on existing behaviour. 43 recorded reference alignments (seeded random log-softmax emissions, including heavy-repeat targets and tight T == L + repeats fits) are bit-identical before and after, as are the existing test_forced_align expectations.
  • No change on a real workload. Aligning a 737 s speech recording against its transcript with a wav2vec2 CTC model (36 820 frames, 10 044 target tokens): the tie condition fires 55 921 times out of 740 M cells, but never on the winning path, and all 2 253 word timestamps are identical before and after. Users on ordinary continuous emissions should see no difference at all.
  • Two independent implementations agree. A scalar Python port of this loop matches stock 234/234 with the clause and matches the fixed kernel 234/234 without it; a separate NumPy Viterbi matches the fixed kernel on 277/277 cases, against 264/277 for stock.
  • Built from main (4e3e282) with USE_CUDA=0 USE_ROCM=0 BUILD_SOX=0 USE_FFMPEG=0; the whole of test/torchaudio_unittest/functional/functional_cpu_test.py passes — 700 passed, 1 xfailed — including the 4 added cases. The CUDA kernel is changed identically but I have no CUDA device to test on; the added test lives in Functional, which functional_cuda_test.py also instantiates, so CI covers both backends.

About the added test

It asserts the score, not the path. Whenever this bug can fire, x1 == x2 means two distinct paths reach that cell with equal score, so an optimum is never unique — which of the equally-good paths is returned is a tie-breaking convention, and pinning it in a test would only make the test brittle.

Note

This touches forced_align_impl in the same region as #4209 (32-bit index overflow, approved and unmerged). The two changes are independent — that one replaces the DP buffers, this one changes a condition — but whichever lands second will want a trivial rebase.

The per-cell choice in forced_align_impl is

    if (x2 > x1 && x2 > x0)      { result = x2; backPtr = 2; }
    else if (x1 > x0 && x1 > x2) { result = x1; backPtr = 1; }
    else                         { result = x0; backPtr = 0; }

When x1 == x2 and both exceed x0, neither condition holds and the else
branch takes x0 -- the strictly worst of the three. The returned path is
then not a most-likely one, which is the entire contract of the function.

Reaching the second branch already implies that x2 is not strictly greater
than both others, so `x1 > x0` alone is the correct and complete condition
there; `&& x1 > x2` only diverts exact ties into the wrong branch. Both
kernels carry the same chain, so both are corrected.

Exhaustively verified on 3243 randomly generated small cases (T <= 5,
vocabulary <= 4, integer log-probabilities so ties are exact) by comparing
against the best score over all enumerated valid paths: 92 sub-optimal
before, 0 after, worst deficit 3.0 log-probability. The existing
forced_align tests are unaffected, as are 43 recorded alignments from real
wav2vec2 emissions -- on non-degenerate emissions exact ties are rare, which
is presumably why this survived.

The added test is asserted on the score rather than on the path: whenever
this bug can fire, x1 == x2 means two distinct paths reach that cell with
the same score, so the optimum is never unique and the choice among optimal
paths is a convention the test should not pin.
@happyarts
happyarts requested a review from a team as a code owner August 23, 2026 00:00
@pytorch-bot

pytorch-bot Bot commented Aug 23, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/4222

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed label Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

forced_align returns a sub-optimal path when two predecessor states tie

1 participant