DES-009: the TTS image builds from our fork until upstream takes the leak fix - #16
Merged
Conversation
added 3 commits
August 15, 2026 22:10
…leak fix docker/tts.upstream moves to secretzer0/Chatterbox-TTS-Server@09a22c75 -- upstream main plus one line, torch.inference_mode around generate. Without it their conds cache pins ~200 MB of autograd graph per distinct voice (measured: 6 voices +1341 MB, guarded +1 MB; ~20 voices OOMed an 8 GB card at 7.39 GiB allocated). Upstream PR devnen/Chatterbox-TTS-Server#166 carries the same line; the pin returns to upstream when it merges. TTS_IMAGE 0.2.1.
…chunks) Cherry-picked upstream PR #156 (three commits, authorship kept) onto the fork: markdown divider lines (---, ***, ___) become paragraph breaks in chunk_text_by_sentences, so a bus message full of "---" no longer glues into one chunk; an optional hard-limit fallback stays off by default. Their regression test passes in the image. Our #166 was closed as a duplicate of upstream #164 (same line); the fork's guard commit is that line. Pin moves to the new fork head; TTS_IMAGE stays 0.2.1 (unpublished).
…rbo in one pass Vetted on an RTX 4070, same 6-chunk message, both paths warmed, 3 runs each: sequential 6.42s, batched (4) 3.79s, batched (8) 3.28s -- 1.7x / 1.96x, the author's claim holds. Cherry-picked #160 (LRU conds cache, its dependency) and #161's two commits with authorship kept, plus one fork commit guarding the batched path with inference_mode: 12 new voices through it grew VRAM 190 MB unguarded, 6 MB guarded, speed unchanged, zero fallbacks. Seeded requests and non-turbo models fall back to the sequential path by construction. Compose sets TTS_BATCH_SIZE=8 for the 12 GB target. Pin moves to fork head 56e59fc.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Operator's call: build the TTS image from secretzer0/Chatterbox-TTS-Server@09a22c75 (upstream 915ae289 + one line: torch.inference_mode around generate) so the image we run does not leak ~200 MB of VRAM per distinct voice. Upstream PR: devnen/Chatterbox-TTS-Server#166 -- pin moves back to devnen when it merges.
Evidence (RTX 4070 8 GB, cu128 image): unguarded, 6 voices +1341 MB allocated with embedding.grad_fn set; guarded, +1 MB. Patched server: 21 clones + 5 predefined + long text, 4414 -> 5256 MB and flat.
docker/tts.upstream is the only input; TTS_IMAGE 0.2.0 -> 0.2.1 (image-pin-check pairs them). Publish will build 0.2.1 cold from the fork SHA on merge. Merge deploys nothing; the voices profile stays down on the VM until the GPU carve-out.