Skip to content

bug(subtitle): missing sentence boundaries make compact mode unreadable with bing-translator #363

Description

@jiangzhuo

Summary

With bing-translator as the translation model, translated segments frequently run
together in the subtitle overlay's compact mode. Sentence boundaries disappear and
long CJK translations become a single unbroken wall of text that is very hard to read.

Reproduced with the local-inference pipeline: cohere-transcribe-webgpu (ASR) +
bing-translator (translation), zh → ja, subtitle overlay in compact mode.

Evidence

From a real session log. Two consecutive translated segments:

…この一票は、Qualcomm または MTK が提供したゴールドサンプルのチップを使用していて、
彼らの性能はテスト販売用の小売版よりも優れています      <- no terminal 。
これらの金のサンプル啊。                                <- next segment

In compact mode these are concatenated into one flowing line, so the reader sees:

…彼らの性能はテスト販売用の小売版よりも優れています これらの金のサンプル啊。

The only boundary is a single half-width space, which in Japanese/Chinese text reads as
no boundary at all.

Contributing causes

There are three, and they compound:

  1. ASR emits zero punctuation. Every local.asr.end from cohere-transcribe-webgpu
    is one unbroken run, e.g.
    好我们就给大家继续拆开用专业设备基于国标好好的测一下各家电池的… (100+ chars, no
    punctuation anywhere). The translator therefore receives no segmentation hints.

  2. Bing is plain MT and cannot be prompted. BingTranslatorClient.translate()
    (src/lib/bing-translator/BingTranslatorClient.ts:98) POSTs the raw text to
    /ttranslatev3 and nothing else. bing-translation.worker.ts:15 declares a
    systemPrompt field on the translate message but never reads it — so the
    "Drop fillers … Fix stuttering and repetitions" instruction that the pipeline logs
    is a no-op for this provider. Consequences observed in the same log:

    • terminal punctuation is inconsistently omitted (some segments end with ,
      others end bare);
    • Chinese fillers leak into the target language:
      这些黄金样品啊。これらの金のサンプル啊。
  3. Compact mode joins segments with a bare space. SubtitleStream.tsx:200-207
    renders each segment as a CompactSpan with leadingSpace={idx > 0}, i.e. adjacent
    segments in a band are separated by exactly one space character. That is adequate for
    Latin scripts and effectively invisible in CJK.

Note this is not strictly Bing-specific — any provider whose output lacks terminal
punctuation hits the same rendering problem. Bing is simply the reliable reproducer,
because unlike the LLM-backed providers it cannot be instructed to punctuate.

Suggested directions

Not mutually exclusive; (a) is the smallest fix, (c) is the most correct:

  • (a) Stronger separator in compact mode. When joining segments, use a visible
    separator instead of a plain space — a wider gap, a styled divider span, or a
    script-aware choice (ideographic space / for CJK, normal space otherwise).
    Purely presentational, no change to stored text.
  • (b) Normalize terminal punctuation per segment. Before rendering, append a
    target-language-appropriate terminator ( for zh/ja, . for Latin scripts) when a
    finalized segment ends without one. Cheap, but mutates displayed text and can be wrong
    when a segment is genuinely mid-sentence.
  • (c) Punctuation restoration on the ASR side. Fixes the root cause and benefits the
    source-language subtitle line too (which today is also completely unpunctuated), but
    is the largest piece of work.

Separately worth cleaning up: the unused systemPrompt field in
bing-translation.worker.ts is misleading — the pipeline logs a prompt that has no
effect on the output.

Environment

  • Provider: local inference (WASM), ASR cohere-transcribe-webgpu, translation
    bing-translator
  • Language pair: zh → ja
  • Surface: subtitle overlay, compact display mode

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions