How do I ensure that it is actually running with CUDA GPU acceleration?

[AbhishtGarg]

CUDA version - 13.0
Driver version - 581.08
GPU - NVIDIA GeForce RTX 5060 Ti

Faster Whisper runs without errors when ran with CUDA, but probably GPU acceleration does not activate. Inference time is identical to CPU execution, and VRAM usage remains zero, indicating that CUDA is not being utilized.

[Purfview]

How do I ensure that it is actually running with CUDA GPU acceleration?

model = WhisperModel(model_size, device="cuda")

Inference time is identical to CPU execution

Did you tested long audio?

Can you post your script?

[AbhishtGarg]

No, tested on multiple small audios of 2-3 secs loaded in a folder.

Below is my script -

I chose device as CUDA.

def run_faster_whisper(device):
from faster_whisper import WhisperModel
start = time.time()
if device == "cuda" and not torch.cuda.is_available():
print("GPU not available")
return
print_memory(f"[{device}] BEFORE MODEL LOAD")
t0 = time.time()
model = WhisperModel("small.en", device=device, compute_type="float32" if device=="cuda" else "float32")
print(f"{device} model load time: {time.time()-t0:.2f}s")
print_memory(f"[{device}] AFTER MODEL LOAD")
files = list_audio_files(AUDIO_FOLDER)
if not files:
print("No audio files")
return
for idx, path in enumerate(files):
audio, sr = librosa.load(path, sr=16000)
segments, info = model.transcribe(audio, beam_size=5, language="en")

print(f"[{device}] TOTAL EXECUTION TIME: {time.time()-start:.2f}s")``

[Purfview]

No, tested on multiple small audios of 2-3 secs loaded in a folder.

Ah, you need to test much longer audios.
For 2-3 secs the model loading/conversion & ect can take more time than actual inference.

BTW, librosa is not needed, you can input a file directly.