whisper.transcribe() vs whisperX.transcribe()

[filmo]

Right now it seems that whisperX is being used exclusively for the better alignment it provides via the Wav2Vec alignment model.

Is there a reason whisperX also isn't used to generate the initial transcription? (via whisperX.transcribe() which seems to return the same result Dict as whisper.transcribe() )

[filmo]

A little bit of digging and I'm going to assume that it's because WhisperModel from faster_whisper is basically 'faster' than the version inside of WhisperX ??

Secondarily, any thoughts on if faster_whisper (as implemented) is better than the HuggingFace implementation for long form audio (more than 5-minute files)

[MahmoudAshraf97]

the transcripe functionality of whisperx using faster_whisper as a backend was introduced in a recent version along with other changes that included removing the the tokens that failed to align instead of keeping the original timestamps, which introduced errors further down the pipeline. So as for now we are pinned to an older version of whisperx until I find a solution for this issue, but anyways the whisperx transcription isn't much faster than the current implementation (around 10% if tuned correctly), and yes faster whisper is a lot faster than the hugging face implementation