Skip to content

Commit cb9d8c3

Browse files
meizhong986claude
andcommitted
docs: v1.8.13 release notes — model revert + scoped WhisperSeg default
Updates the v1.8.13 release notes to honestly reflect the post-F4/F6/F7 acceptance testing findings: Title and theme: - Adds "Model Default Revert" to the title alongside the WhisperSeg Ensemble/Qwen/Decoupled scoping. - Theme paragraph explicitly mentions both the WhisperSeg scope reduction and the large-v3 → large-v2 revert. New "Mode-by-mode default policy" table clarifies which path gets which segmenter default (Ensemble/Qwen/Decoupled get whisperseg; simple modes get silero-v3.1) and links to the Known caveats. Known caveats section gains two entries: 1. Default Whisper model reverted from large-v3 to large-v2 — full empirical context: F4/F6/F7 catastrophic (6-10 entries / 68 GT) vs F8 with large-v2 (51 entries / 68 GT). Root cause: v1.8.12 aggressive preset retune was tuned against large-v2 forensic acceptance data. Users wanting v3 can opt in via --model large-v3. References the diagnostic suite test artifacts at test_media/1813 acceptance/F4/DIAG_FW/chunk_length_test/. 2. WhisperSeg in simple Transcription Mode deferred to v1.9.0 — same explanation as before, retained for context. Internal changes section gains a "Default model revert" sub-bullet listing all three modified file locations: - config/components/asr/faster_whisper.py:211 - config/components/asr/stable_ts.py:293 - webview_gui/assets/index.html:299 v1.9.0 follow-up section adds a new marquee item: - Re-tune aggressive sensitivity preset for large-v3 (lifts the v1.8.13 model revert). Plan: re-run forensic acceptance suite against large-v3, produce per-engine, per-model preset variants. User-facing how-to-opt-into sections updated: - "How to opt into WhisperSeg in simple mode" → use --ensemble - "When NOT to use WhisperSeg" — non-Japanese audio, CPU-bound, v1.8.12 retest cluster File: docs/release_notes_v1.8.13.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 745f970 commit cb9d8c3

1 file changed

Lines changed: 135 additions & 30 deletions

File tree

docs/release_notes_v1.8.13.md

Lines changed: 135 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,84 @@
1-
# WhisperJAV v1.8.13 — Polish + WhisperSeg Default
1+
# WhisperJAV v1.8.13 — Polish + WhisperSeg Default (Ensemble / Qwen / Decoupled) + Model Default Revert
22

33
> Draft — to be finalized before tag.
44
55
## Theme
66

77
A focused polish release that addresses long-standing paper cuts, fixes
8-
the install regression carried over from v1.8.11, and **promotes WhisperSeg
9-
from opt-in to default** speech segmenter across all surfaces.
8+
the install regression carried over from v1.8.11, **promotes WhisperSeg
9+
to the default speech segmenter on the pipeline paths that fully support
10+
it** (Ensemble, Qwen, Decoupled), and **reverts the default Whisper model
11+
from `large-v3` back to `large-v2`** for both the FasterWhisper (BalancedPipeline)
12+
and Stable-TS (Fast/Faster pipelines) backends, restoring transcription
13+
quality on continuous-energy non-phonetic JAV content. Simple Transcription
14+
Mode keeps silero-v3.1 in v1.8.13, pending a config-routing refactor
15+
scheduled for v1.9.0. See "Known caveats" for full context on the model
16+
revert.
1017

1118
---
1219

13-
## Headline change: WhisperSeg is now the default speech segmenter
20+
## Headline change: WhisperSeg becomes the default for Ensemble / Qwen / Decoupled
1421

1522
Introduced in v1.8.12 as opt-in, **WhisperSeg ONNX is now the default
16-
speech segmenter** in v1.8.13. On the Netflix-GT 283s JAV reference
23+
speech segmenter** for Ensemble Mode (pass1 and pass2), the Qwen pipeline,
24+
and the Decoupled pipeline. On the Netflix-GT 283s JAV reference
1725
benchmark it scored F1=0.787 — the strongest result among the four
1826
shipped backends:
1927

2028
| Segmenter | F1 (Netflix-GT JAV) |
2129
|---|---:|
22-
| **WhisperSeg (new default)** | **0.787** |
30+
| **WhisperSeg (new default for Ensemble/Qwen/Decoupled)** | **0.787** |
2331
| TEN VAD | 0.698 |
2432
| Silero v6.2 | 0.654 |
25-
| Silero v3.1 (previous default) | 0.625 |
33+
| Silero v3.1 (default for simple Transcription Mode) | 0.625 |
34+
35+
### Mode-by-mode default policy in v1.8.13
36+
37+
| Path | Default segmenter | Why |
38+
|---|---|---|
39+
| **GUI Ensemble Mode** (pass1 + pass2) | WhisperSeg | Pass-worker routes grouping params correctly |
40+
| **`--ensemble` CLI** | WhisperSeg | Same path as above |
41+
| **`--mode qwen`** (ChronosJAV) | WhisperSeg | QwenPipeline forwards `chunk_threshold_s` / `max_group_duration_s` explicitly |
42+
| **`--pipeline decoupled`** | WhisperSeg | DecoupledPipeline kwargs path |
43+
| **GUI Transcription Mode** | **silero-v3.1** | Routes through legacy ASR constructors with a known param-routing bug; fix is v1.9.0 |
44+
| **`--mode balanced` / `--mode fidelity`** (without `--ensemble`) | **silero-v3.1** | Same — see Known caveats below |
2645

2746
### What changes for you
2847

29-
- First transcription after upgrade downloads ~119 MB WhisperSeg model
30-
from HuggingFace (one time). Cached at `~/.cache/huggingface/` (Linux/macOS)
31-
or `%USERPROFILE%\.cache\huggingface\` (Windows).
32-
- ChronosJAV pipeline's segmenter default also flipped (was silero-v6.2).
33-
- Colab + Kaggle expert notebook defaults flipped to match.
48+
- **Ensemble Mode users** get WhisperSeg out of the box on first run. First
49+
transcription downloads ~119 MB WhisperSeg model from HuggingFace
50+
(one time). Cached at `~/.cache/huggingface/` (Linux/macOS) or
51+
`%USERPROFILE%\.cache\huggingface\` (Windows).
52+
- **Simple Transcription Mode users** see no behavior change vs v1.8.12 —
53+
silero-v3.1 stays the default.
54+
- ChronosJAV (`--mode qwen`) segmenter default flipped to WhisperSeg.
55+
- Colab + Kaggle expert notebook defaults flipped to match (Ensemble path).
56+
57+
### How to opt into WhisperSeg in simple mode
3458

35-
### When NOT to use WhisperSeg
59+
Until the v1.9.0 routing fix lands, the recommended path to use WhisperSeg
60+
is **Ensemble Mode** (which also gives you the dual-pass merging that has
61+
become WhisperJAV's strongest result on JAV content). In the GUI, switch
62+
from the "Transcription Mode" tab to the "Ensemble Mode" tab. On the CLI,
63+
add `--ensemble` plus pass1/pass2 settings.
64+
65+
If you explicitly pass `--speech-segmenter whisperseg` (or `ten`, `nemo`,
66+
`whisper-vad`) to a simple `--mode balanced` / `--mode fidelity` run,
67+
v1.8.13 will warn loudly and downgrade to silero-v3.1 to avoid catastrophic
68+
empty output. The warning includes a pointer to `--ensemble`.
69+
70+
### When NOT to use WhisperSeg (Ensemble path)
3671

3772
- **Non-Japanese audio** (Korean, Chinese, English): WhisperSeg is trained
3873
on Japanese ASMR. For other languages, override:
39-
- **CLI**: `--speech-segmenter silero-v3.1`
74+
- **CLI**: `--pass1-speech-segmenter silero-v3.1 --pass2-speech-segmenter silero-v3.1`
4075
- **GUI Ensemble tab**: pick "Silero v3.1" from the speech segmenter
41-
dropdown
76+
dropdown for each pass
4277
- **CPU-bound users**: WhisperSeg is ~2-3× slower than silero/ten on CPU.
4378
If you have no GPU (or onnxruntime-gpu isn't installed), silero-v3.1
4479
may produce faster total runtime.
4580
- **v1.8.12 retest cluster** (#294, #302, #287, #297): if you're validating
46-
the v1.8.12 ASR fixes, keep `--speech-segmenter silero-v3.1` for clean
81+
the v1.8.12 ASR fixes, keep silero-v3.1 in both passes for clean
4782
attribution of which fix helped your case.
4883

4984
---
@@ -122,13 +157,52 @@ Re-open the notebook from the [latest commit](https://github.com/meizhong986/Whi
122157

123158
## Known caveats
124159

125-
- **Whisperseg field testing is thin**. v1.8.12's WhisperSeg release got
126-
zero user reports back. v1.8.13's promotion to default rests on the
127-
Netflix-GT bench plus internal smoke tests. If you observe regressions
128-
vs v1.8.12, override to silero-v3.1 and report on the issue tracker.
129-
130-
- **Cold-start cost**: first transcription after upgrade pauses for the
131-
119 MB WhisperSeg model download. Subsequent runs use the cached model.
160+
- **Default Whisper model reverted from `large-v3` to `large-v2`.** Late in
161+
v1.8.13 acceptance testing (F4/F6/F7 vs F8 side-by-side on the same
162+
293-second JAV reference clip), the v1.8.12 aggressive ASR preset retune
163+
(`no_speech_threshold=0.84`, `beam_size=3`, `best_of=2`,
164+
`temperature=[0.0, 0.17]`, `compression_ratio_threshold=2.6`,
165+
`repetition_penalty=1.3`, `no_repeat_ngram_size=3`, `chunk_length=30`)
166+
was found to interact pathologically with `large-v3` on continuous-energy
167+
non-phonetic content (JAV moaning), producing **6–10 SRT entries out of
168+
68 ground-truth** (≈85–90% loss) under simple Transcription Mode. The
169+
same audio + same preset values + `large-v2` produced **51 entries**
170+
(≈75% capture). Root cause: the v1.8.12 retune was tuned against
171+
large-v2 forensic acceptance data; large-v3's slightly different
172+
encoder/decoder behavior makes the same gate values too strict on this
173+
content distribution. Until the preset is re-tuned for large-v3 in
174+
v1.9.x, v1.8.13 reverts the default to large-v2 across both
175+
FasterWhisper (`config/components/asr/faster_whisper.py`) and Stable-TS
176+
(`config/components/asr/stable_ts.py`) backends. Users who want
177+
large-v3 can opt in with `--model large-v3` (CLI) or via the GUI model
178+
override checkbox. OpenAI-Whisper backend (FidelityPipeline) was
179+
already on large-v2 in v1.8.12 and is unaffected.
180+
181+
- **WhisperSeg in simple Transcription Mode is deferred to v1.9.0.**
182+
During F4/F6 acceptance testing, WhisperSeg invoked through the simple
183+
`--mode balanced` path produced catastrophic empty output (10 of 68
184+
ground-truth subtitles) on JAV moaning content, while the same audio +
185+
same backend through Ensemble Mode produced 52/68. Root cause is a
186+
config-routing path: BalancedPipeline / FidelityPipeline use a
187+
CONSTRUCTOR FIREWALL in their ASR modules that strips backend-agnostic
188+
grouping params (`chunk_threshold_s`, `max_group_duration_s`) for
189+
non-Silero backends, causing WhisperSeg to fall back to its 29-second
190+
default group duration and trigger a Whisper repetition pathology.
191+
v1.8.13 ships the SAFE scope (Ensemble + Qwen + Decoupled keep
192+
WhisperSeg, simple modes keep silero-v3.1). v1.9.0 lands the proper
193+
fix (split SileroVADOptions, introduce SegmenterGroupingOptions,
194+
eliminate the firewall pattern).
195+
196+
- **WhisperSeg field testing is thin**. v1.8.12's WhisperSeg release got
197+
zero user reports back. v1.8.13's promotion to default for the Ensemble
198+
path rests on F5 acceptance test (52/68 GT), the Netflix-GT bench, plus
199+
internal smoke tests. If you observe regressions vs v1.8.12, switch
200+
back to silero-v3.1 in the Ensemble dropdown and report on the issue
201+
tracker.
202+
203+
- **Cold-start cost**: first Ensemble transcription after upgrade pauses
204+
for the 119 MB WhisperSeg model download. Subsequent runs use the
205+
cached model.
132206

133207
- **Customize Parameters modal**: the Anime-Whisper model dropdown shows
134208
v0.3 as an option but defaults to litagin/anime-whisper. v0.3 is
@@ -155,14 +229,25 @@ Re-open the notebook from the [latest commit](https://github.com/meizhong986/Whi
155229

156230
## Internal changes
157231

158-
- WhisperSeg promotion touched 11 locations across resolver, ASR module
232+
- WhisperSeg promotion touched 11+ locations across resolver, ASR module
159233
fallbacks, qwen pipeline, CLI argparse help, GUI HTML, decoupled YAML,
160-
and Colab + Kaggle notebooks. Architectural note added inline in
161-
`legacy.py` explaining why `LEGACY_PIPELINES["balanced"|"fidelity"]["vad"]`
162-
field stays at "silero-v3.1" (it names a Pydantic VAD *component* that
163-
defines preset values, not the runtime segmenter backend selector — the
164-
firewall in `whisper_pro_asr.py:71-77` clears those silero presets at
165-
runtime when whisperseg is selected).
234+
GUI app.js per-pipeline presets, and Colab + Kaggle notebooks.
235+
236+
- **Default model revert (large-v3 → large-v2)** in three locations:
237+
`config/components/asr/faster_whisper.py:211`,
238+
`config/components/asr/stable_ts.py:293`, and
239+
`webview_gui/assets/index.html:299` (GUI model dropdown default selection).
240+
Inline comments reference `faster_whisper.py` for full empirical context.
241+
OpenAI-Whisper backend (`config/components/asr/openai_whisper.py:178`)
242+
was already on large-v2 and unchanged.
243+
244+
- **v1.8.13 default-flip scope reduction** (post-F4/F6): `main.py`'s
245+
`--speech-segmenter` default-resolution gained an explicit allow-list of
246+
paths that route segmenter grouping params correctly to non-Silero
247+
backends (`--ensemble`, `--pipeline decoupled`, `--mode qwen`). Other
248+
paths default to silero-v3.1. Explicit `--speech-segmenter whisperseg`
249+
on a non-allow-listed path emits a hard warning and downgrades to
250+
silero-v3.1. See inline comment at `main.py:1840-1898`.
166251

167252
- 6 prepared post-release replies in
168253
`docs/release_v1.8.13_reply_drafts.md` will be posted after the GitHub
@@ -174,6 +259,26 @@ Re-open the notebook from the [latest commit](https://github.com/meizhong986/Whi
174259

175260
Marquee features for v1.9.0 (per `docs/plans/PRODUCT_VISION_AND_ROADMAP_v1.9_v2.md`):
176261

262+
- **Re-tune aggressive sensitivity preset for `large-v3`** (lifts the
263+
v1.8.13 model revert). The current v1.8.12 aggressive preset values
264+
(`no_speech_threshold=0.84`, `beam_size=3`, etc.) were tuned against
265+
large-v2 forensic acceptance data and produce catastrophic empty
266+
output on JAV content with large-v3. v1.9.x will re-run the forensic
267+
acceptance suite against large-v3 and produce a per-engine, per-model
268+
preset variant (e.g., `aggressive_v3` with relaxed gates) so large-v3
269+
can return as the default. Reference: F4/F6/F7 vs F8 acceptance test
270+
artifacts in `test_media/1813 acceptance/`.
271+
272+
- **Unified segmenter param routing** (lifts the v1.8.13 simple-mode
273+
WhisperSeg restriction). Plan: split `SileroVADOptions` into
274+
`SileroVADOptions` (Silero-specific) + `SegmenterGroupingOptions`
275+
(backend-agnostic: `chunk_threshold_s`, `max_group_duration_s`,
276+
`max_speech_duration_s`); resolver places grouping params in a canonical
277+
location all consumers read from; eliminate the constructor firewall in
278+
`faster_whisper_pro_asr.py` and `whisper_pro_asr.py`. After this,
279+
WhisperSeg becomes the default everywhere including simple Transcription
280+
Mode. Reference: F4/F6 acceptance test artifacts and the diagnostic
281+
suite runs at `test_media/1813 acceptance/F4/DIAG_FW/`.
177282
- **GUI redesign**: 5→4 tabs, eliminate Advanced, add Utilities tab
178283
- **Standalone Merge utility** (GUI + `whisperjav-merge` CLI) — preview at
179284
Section 8.5 of the roadmap doc

0 commit comments

Comments
 (0)