Compatibility with turbo-large-v3

[moumeneb98]

I tried to use large v3 turbo in tensorrt_llm but it seems it's not compatible, was anyone able to run it ?

[sh1man999]

@moumeneb98 Have you tried running it? https://huggingface.co/deepdml/faster-whisper-large-v3-turbo-ct2

[moumeneb98]

It works in Ctranslate2 but not in tensorrt

[sh1man]

Isn't Ctranslate2 (turbo-large-v3) faster now ?

[sh1man]

if you still need tensorrt_llm, https://github.com/k2-fsa/sherpa/tree/master/triton/whisper

[FerLuisxd]

+1 to this

[muhammad-faizan-122]

@sh1man , can you please guide me how can utilize TensorRT backend rather than python backend?

[rootint]

You can add turbo support by adding the links to the model weights in download_utils.py:

_MODELS = {
    "large-v2": "https://openaipublic.azureedge.net/main/whisper/models/81f7c96c852ee8fc832187b0132e569d6c3065a3252ed18e56effd0b6a73e524/large-v2.pt",
    "large-v3": "https://openaipublic.azureedge.net/main/whisper/models/e5b1a55b89c1367dacf97e3e19bfd829a01529dbfdeefa8caeb59b3f1b81dadb/large-v3.pt",
    "large-v3-turbo": "https://openaipublic.azureedge.net/main/whisper/models/aff26ae408abcba5fbf8813c21e62b0941638c5f6eebfb145be0c9839262a19a/large-v3-turbo.pt",
}

_TOKENIZER = {
    "large-v2": "https://huggingface.co/Systran/faster-whisper-large-v2/raw/main/tokenizer.json",
    "large-v3": "https://huggingface.co/Systran/faster-whisper-large-v3/raw/main/tokenizer.json",
    "large-v3-turbo": "https://huggingface.co/deepdml/faster-whisper-large-v3-turbo-ct2/resolve/main/tokenizer.json",
}