Skip to content

fix(tts): prevent runaway GPT-SoVITS filler vowels - #22

Merged
Lucas1479 merged 3 commits into
mainfrom
codex/gsv-semantic-stability
Sep 1, 2026
Merged

fix(tts): prevent runaway GPT-SoVITS filler vowels#22
Lucas1479 merged 3 commits into
mainfrom
codex/gsv-semantic-stability

Conversation

@Lucas1479

@Lucas1479 Lucas1479 commented Aug 31, 2026

Copy link
Copy Markdown
Member

中文摘要

给中文审阅者的速览;下方英文正文保留完整实验数据。

  • 问题:GPT-SoVITS 在静态 KV cache 和 CUDA Graph 路径下可能读取尚未写入或仅用于对齐的缓存位置,导致语义 token 坍缩为长串重复元音,最终产生失控的填充音。
  • 修复:屏蔽无效 KV 位置;从真实 prompt 长度连续生成;bucket 溢出时保留完整条件上下文;在进入 SoVITS/BigVGAN 前拦截与 token ID 无关的坍缩候选,并只允许一次有界、可观测的重试;同时隔离 Graph capture 的随机数状态并使修复前首句音频缓存失效。
  • 实验结果:固定 660 次、同 GPU/模型/文本/种子的对照中,最终 Graph+static、static、dynamic 三条路径的异常均从修复前的 24/220、24/220、14/220 降为 0/220;220/220 组语义流与保护决策保持一致。示例语音从 9.77 秒、最长 192 个重复 token,改善为 5.04 秒、最长重复 10 个 token,ASR 能保留完整续句。
  • 性能与审阅重点:Graph/static 中位延迟仍为 108.4 ms,约比非 Graph 路径快 2.75 倍。建议重点审阅保护规则的误杀风险、单次重试的可观测性、随机数隔离以及缓存版本失效是否完整。
  • 验证:50 项公开 TTS/backend 定向测试、语义保护 fail-closed 测试、第三方来源门禁、源码发布门禁、Python 编译与 diff 检查均通过。

关联问题:#21

Summary

  • mask unwritten and CUDA-Graph alignment-gap KV positions while preserving fixed graph shapes
  • generate contiguously from the real prompt length and preserve full conditioning context on bucket overflow
  • reject token-ID-independent collapsed semantic candidates before SoVITS/BigVGAN, with one bounded observable retry
  • isolate graph capture from sampling RNG and invalidate pre-fix first-sentence audio-cache identities
  • add paired-seed stability probes, real-audio rendering support, static/dynamic parity tests, and guard/cache regression tests

Evidence

A fixed 660-trial filler-heavy paired-seed cohort (same physical GPU, model hashes, texts, seeds, and 400-token cap) changed from:

Phase Graph + static Static Dynamic
Before 24/220 24/220 14/220
KV fix only 14/220 14/220 14/220
Final guarded path 0/220 0/220 0/220

After the KV fix, Graph/static and static/dynamic semantic streams match exactly for 220/220 paired case/seed combinations. The final guard makes the same retry decision in all 220/220 pairs.

For うーん……正直、まだ完全には分からないわ。 at seed 2001:

  • before: 243 semantic tokens, longest equal-token run 192, 9.77 s WAV, ASR only うーん。
  • after: 125 tokens, longest run 10, 5.04 s WAV, ASR retains the complete continuation

Normal median T2S latency on the final full matrix is 108.4 ms Graph/static, 300.0 ms static, and 297.8 ms dynamic; Graph remains about 2.75x lower-latency than non-Graph paths.

Validation

  • 50 passed targeted public TTS/backend regression suite
  • direct project-environment semantic guard retry/fail-closed tests passed
  • third-party provenance release gate passed
  • deterministic source-release gate passed (0 errors, 0 warnings)
  • Python compilation and git diff --check passed

Closes #21

@Lucas1479 Lucas1479 added bug Something isn't working python Pull requests that update python code labels Aug 31, 2026
@Lucas1479
Lucas1479 force-pushed the codex/gsv-semantic-stability branch from e589dcf to 7bb99b3 Compare September 1, 2026 06:26
@Lucas1479

Copy link
Copy Markdown
Member Author

Maintainer self-review complete: no blocking findings. The decoder masks unwritten/alignment slots, writes from the real prompt length, and preserves full conditioning by switching to dynamic KV at bucket exhaustion. The pre-vocoder guard is token-ID independent, retries at most once, and fails closed; Graph capture no longer advances sampling RNG; the first-sentence cache revision prevents reuse of pre-fix audio. Validation on the rebased head in the formal .venv_cu124 environment (Python 3.12.10, Torch 2.5.1+cu124): 59 focused TTS/cache/Graph/guard tests passed, changed-surface Ruff/compile/diff checks passed, provenance passed with 0 errors, and deterministic source-release passed with 0 errors/0 warnings. A fresh physical-GPU paired probe (2 seeds x 4 quick cases x 3 modes = 24 trials, production one-retry guard) completed with 0 anomalies, 0 errors, 100% EOS, and 100% Graph/static and static token parity against dynamic. Required build and cpu-model-less checks are green on head 7bb99b3. Admin merge bypass is used only for the unavailable external approval requirement.

@Lucas1479
Lucas1479 merged commit 9c7ca4a into main Sep 1, 2026
2 checks passed
@Lucas1479
Lucas1479 deleted the codex/gsv-semantic-stability branch September 1, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GPT-SoVITS can produce runaway filler vowels with static KV cache

1 participant