STT Sometimes does not produce audio despite being provided audio ? and reset doesn't reset the time stamps ?

[ArEnSc]

Backend impacted

The Rust implementation

Operating system

OSX

Hardware

CPU

Description

I wonder if it could be due to noise, but there are times where in the model doesn't produce tokens, I wonder if this has to do with high noise scrambling the model? for a short time and reset not functioning correctly? I am using it in a streaming scenario

Extra information

Its a bit difficult to reproduce, I wonder if you have encountered it a few times

Environment

Fill in the following information on your system.

• Operating system version:

If the backend impacted is PyTorch:

• Python version:
• PyTorch version:
• CUDA version (run python -c 'import torch; print(torch.version.cuda)'):
• GPU model and memory:

If the backend is MLX:

• Mac model:

[ArEnSc]

@LaurentMazare

[LaurentMazare]

We've indeed observed something similar with the 1b model. It's not very common but from time to time it stops transcribing for a long period, we haven't found a specific way to trigger this failure mode though. The 2.6b model seems much more robust to this so if you don't need the VAD or fr support, you may consider switching to it (the quality of the transcription is also better).

[ArEnSc]

We've indeed observed something similar with the 1b model. It's not very common but from time to time it stops transcribing for a long period, we haven't found a specific way to trigger this failure mode though. The 2.6b model seems much more robust to this so if you don't need the VAD or fr support, you may consider switching to it (the quality of the transcription is also better).

Is there a way to reset the model after each utterance to ensure this mode of failure wouldn't happen? I suspect maybe its high noise and then this enters the context of the model ? I guess I could clear the context after each utterence

[LaurentMazare]

Is there a way to reset the model after each utterance to ensure this mode of failure wouldn't happen? I suspect maybe its high noise and then this enters the context of the model ? I guess I could clear the context after each utterence

We haven't tried it out so not sure. The simpler would probably be to run multiple inferences in parallel with different contexts (this should be easy with the rust server). If you think high noise is causing this to happen frequently, you may also want to pre-process your audio a bit to clean it up.

[ArEnSc]

Thanks!!!