Hallucinations during music/no Dialogue

[francetoastVN]

WhisperJAV 1.8.10
MacOS
Transcription Mode
Balanced: Aggressive, Direct to English:
1
00:00:21,166 --> 00:00:26,366
Thank you for your viewing. Please subscribe to the channel if it is good. 魔王魂

2
00:00:49,390 --> 00:00:54,590
Thank you for your viewing. Please subscribe to the channel if it is good. 魔王魂

3
00:01:09,310 --> 00:01:11,330
This is the first time I've done this.

4
00:01:11,330 --> 00:01:12,150
It's a little hard to see, but it looks good.

5
00:01:17,756 --> 00:01:20,290
Next time, I'll show you how it works.

6
00:01:20,290 --> 00:01:24,310
Thank you for watching until the end of the video.

7
00:01:34,730 --> 00:01:35,350
Please subscribe to the channel.

8
00:01:59,150 --> 00:02:01,150
This is the end of this video.

9
00:02:01,790 --> 00:02:05,030
Please subscribe to my channel and give it a high rating.

10
00:02:05,030 --> 00:02:08,090
See you in the next video!

11
00:02:46,710 --> 00:02:47,210
Hello?
(Start of phone conversation)

---

Balanced: Balanced, Direct to English:
1
00:02:46,738 --> 00:02:47,238
Hello?
(Start of phone conversation)

No more hallucinations during music-improvement from 1.8.8, which had two 'subscribe to my channel' instances!

---

Balanced: Conservative, Direct to English:
1
00:02:46,710 --> 00:02:47,210
Hello?
(Start of phone conversation)
Conservative now picks up the phone conversation (it ignored the entire conversation in 1.8.8)

[meizhong986]

Thanks for the update.

It seems that 1.8.10 performs better for your case. As for the Aggressive sensitivity, it is the expected behavior, it is prone to hallucination. While it tries to make sense of any possible signal, it picks up noise as signal in some situations.

Coupe of things:

• I have added a new feature in 1.8.10 that does speech enhancement (removes music and noise), from VAD. This means the whisper model will not receive the music parts, and should get more accurate segments of dialogs. It takes more time but so far I have been happy with the result. Give it a try with ZipEnhancer VAD Only option .

• I'll add those English hallucination phrases to the sanitization filter

[francetoastVN]

I'll give it a try-Right now installing that new OLLAMA thing-it sounds promising!
Also, thanks for the easy update install! It works great on the Mac!

[TinyRick1489]

@meizhong986
How to use speech enhancement for VAD in CLI?