Open-source models that turn Italian audio into text, locally, on your own hardware, with optional speaker diarization. Brought to you by the LocalAI team.
🇮🇹 In italiano · leggi qui
Una famiglia di modelli open-source per trascrivere audio in italiano, da eseguire in locale sul proprio hardware, con diarizzazione (etichette parlante) opzionale. Realizzata dal team di LocalAI.
Perché esistono questi modelli. La maggior parte dei modelli Whisper italiani disponibili online sono addestrati solo su Common Voice (persone che leggono frasi al microfono di casa). Vanno benissimo nei benchmark di lettura, ma collassano sull'audio reale: riunioni, telefonate, podcast tornano pieni di ripetizioni allucinate e frasi inventate. Abbiamo addestrato una famiglia completa pensata anche per il parlato spontaneo.
Quale modello scegliere:
| Scenario | Modello consigliato |
|---|---|
| 🎙️ Riunioni, chiamate, podcast, audio reale | whisper-large-v3-turbo-it-yodas-only |
| 📚 Audiolibri, notizie, atti parlamentari | whisper-large-v3-it-multi |
| 🗣️ Servono le etichette parlante (diarizzazione) | vibevoice-asr-it-yodas-only |
Esempio rapido (Python):
pip install faster-whisperfrom faster_whisper import WhisperModel
model = WhisperModel("LocalAI-io/whisper-large-v3-turbo-it-yodas-only-ct2-int8",
device="cpu", compute_type="int8") # oppure device="cuda"
segments, _ = model.transcribe("audio.mp3", language="it")
for s in segments:
print(f"[{s.start:5.1f}s] {s.text}")Pronti per qualunque runtime. Ogni modello è pubblicato in tutti i formati di quantizzazione che la community usa davvero, non devi convertire niente a mano:
safetensorsper 🤗 Transformers (precisione piena, PyTorch).CTranslate2 INT8perfaster-whisper/ WhisperX / LocalAI, circa 4× più piccolo, ottimo su CPU.whisper.cppGGML (f16/q4_0/q5_0/q8_0) per whisper.cpp e LocalAI, perfetto per edge / Raspberry Pi / mobile.- GGUF per la famiglia VibeVoice via vibevoice.cpp.
Tutti i 75 repo HuggingFace sono qui: huggingface.co/LocalAI-io. Tutto sotto licenza MIT, nessuna chiave API, nessuna telemetria, gira completamente in locale.
Per la documentazione completa (tutte le varianti, i benchmark, i dettagli tecnici), continua a leggere in inglese qui sotto. 👇
Pick the path that matches your stack. All models live on the LocalAI-io HuggingFace organization and run fully offline.
Python (faster-whisper) is recommended for most users:
pip install faster-whisperfrom faster_whisper import WhisperModel
model = WhisperModel("LocalAI-io/whisper-large-v3-turbo-it-yodas-only-ct2-int8",
device="cpu", compute_type="int8") # or device="cuda"
segments, _ = model.transcribe("audio.mp3", language="it")
for s in segments:
print(f"[{s.start:5.1f}s] {s.text}")Command line (whisper.cpp), single-binary, CPU or CUDA:
hf download LocalAI-io/whisper-large-v3-turbo-it-yodas-only-ggml \
ggml-model-q5_0.bin --local-dir .
./whisper-cli -m ggml-model-q5_0.bin -l it -f audio.wavLocalAI server: OpenAI-compatible HTTP endpoint. See the LocalAI section below for ready-made YAML configs and curl example.
Need speaker labels? Use the VibeVoice variant; it produces [Speaker 0/1/2…] text segments natively.
A family of Italian-tuned models in two architecture families and three deployment formats:
- Whisper family, every model size (tiny → large-v3-turbo) × four training recipes (CV-only, Multi, Multi+YODAS, YODAS-only). Best raw throughput on GPU.
- VibeVoice ASR family, a Qwen2-based architecture with native speaker diarization for conversational audio. Best when you need speaker labels or fewer hallucinated repetitions.
Half the reason this catalog is large is that we ship each fine-tune in every quantization format the community actually uses, so you don't have to download a PyTorch checkpoint and convert it yourself. Pick the runtime that fits your hardware:
safetensors(full precision), for 🤗 Transformers, research, and PyTorch fine-tuning.CTranslate2 INT8(~4× smaller, CPU-friendly), forfaster-whisper/WhisperX/ LocalAI'swhisperxbackend. Best on CPU and the default for production transcription.whisper.cppGGML in four bit-widths (f16,q4_0,q5_0,q8_0), for whisper.cpp, pywhispercpp, LocalAI'swhisperbackend, mobile / edge / Raspberry-Pi-style deployments.q5_0is the usual sweet spot.- GGUF (
q4_0,q5_0,q8_0) for vibevoice.cpp, covers the VibeVoice family on the same C++/CUDA runtime.
That's a total of 75 repos on HuggingFace, all maintained in lock-step so the q5_0 GGML matches the int8 CTranslate2 build matches the safetensors source.
Why the training data matters. A speech-to-text model is only good at the kind of speech it has heard during training. A model trained on people carefully reading sentences out loud will look fantastic on benchmark numbers and then make up entire sentences when you hand it a real meeting where two humans talk over each other. So we trained four versions of each model on four different "diets" of audio, ranging from clean read speech to messy real-world conversations.
The four datasets we mix and match:
- Common Voice (CV), Mozilla's crowdsourced corpus. People reading sentences on a phone or laptop at home. Clean, varied speakers, but it's read speech, not conversational.
- MLS (Multilingual LibriSpeech), long-form audiobook readings. Professional narrators, no overlap, no hesitations.
- VoxPopuli (VoxPopuli), European Parliament recordings. Formal speech, but real microphones in a real room.
- YODAS-Granary (YODAS), YouTube clips, pseudo-labeled. Real people talking spontaneously: meetings, podcasts, vlogs. Italian as it's actually spoken.
The four mixes:
| Variant | What it's trained on (plain-English) | Best for | Samples |
|---|---|---|---|
CV-only (*-it) |
Just Common Voice. People reading sentences at home. | CV-style benchmark WER. Falls apart on real conversations. | 173k |
Multi (*-it-multi) |
Common Voice + audiobooks + parliament. All read or formal speech. | Audiobooks, news, parliamentary transcripts. Stable across read-speech domains. | 255k |
Multi + YODAS (*-it-multi-yodas-asr_only-nocv) |
Audiobooks + parliament + YouTube spontaneous Italian (~87k clips). | Best when you don't know what kind of audio you'll get. | 164k |
YODAS-only (*-it-yodas-only) |
YouTube spontaneous Italian only (240k clips). 100% real conversational audio. | Real-world meetings, calls, podcasts. Read-speech benchmark numbers regress (no audiobooks in training). | 240k |
Recommended pick:
- 🎙️ Meetings, calls, podcasts, real-world Italian:
*-it-yodas-only(large-v3-turbo is the sweet spot for production) - 📚 Audiobooks, news, parliamentary:
*-it-multi - 🎯 CV-style benchmarks only:
*-it - 🔀 Mixed/unknown audio:
*-it-multi-yodas-asr_only-nocvor*-it-multi - 🗣️ Multi-speaker / need diarization:
vibevoice-asr-it-yodas-only(see VibeVoice section)
Each link below points to a HuggingFace repo. The ggml repos contain four quantizations (f16, q4_0, q5_0, q8_0), pick by your size/quality budget. The newer YODAS-trained ggml repos ship q4_0, q5_0, q8_0 (no f16, it consistently failed to upload via the HF Xet protocol; the q8_0 is near-lossless in practice).
| Base | Params | CV-only (*-it) |
Multi (*-it-multi) |
|---|---|---|---|
openai/whisper-tiny |
39M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-base |
74M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-small |
244M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-medium |
769M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-large-v3 |
1.55B | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-large-v3-turbo |
809M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
| Base | Params | Multi + YODAS (*-it-multi-yodas-asr_only-nocv) |
YODAS-only (*-it-yodas-only) |
|---|---|---|---|
openai/whisper-tiny |
39M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-base |
74M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-small |
244M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-medium |
769M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-large-v3 |
1.55B | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
openai/whisper-large-v3-turbo |
809M | safetensors · ct2-int8 · ggml | safetensors · ct2-int8 · ggml |
Note: smaller
*-yodas-onlysizes (tiny/base/small/medium) are inherently loop-prone on conversational audio compared tolarge-v3/large-v3-turbo. They're shipped for completeness and edge / CPU-only deployments, for the cleanest output on real-world Italian speech, preferlarge-v3-turbo-it-yodas-only.
Word Error Rate (%) on the Common Voice 25.0 Italian test set (15,184 samples). Lower is better.
| Base | Params | CV-only | Multi |
|---|---|---|---|
whisper-tiny |
39M | 27.1% | 29.4% |
whisper-base |
74M | 19.2% | 21.4% |
whisper-small |
244M | 13.0% | 15.6% |
whisper-medium |
769M | 10.5% | 12.4% |
whisper-large-v3 |
1.55B | , | , |
whisper-large-v3-turbo |
809M | , | , |
Large-v3 and large-v3-turbo results: these variants were trained with evaluation disabled (to avoid VRAM fragmentation during generation on very long sequences). Benchmarks will be added once we finish running the standalone eval pass, they are expected to improve on the
whisper-mediumnumbers above.Note on the Multi column: the
*-multimodels are evaluated on the CV-only test set here for an apples-to-apples comparison with the*-itvariants. They were trained on a more diverse mix (CV + MLS + VoxPopuli), so CV-only WER slightly under-represents their real-world robustness on non-Common-Voice audio. If your deployment sees audiobooks, parliamentary recordings, or other non-crowdsourced speech, prefer the*-multivariant.
Training details: 10,000 steps, learning rate 1e-5, 500 warmup steps, bf16 on an NVIDIA GB10 (128GB unified memory). Common Voice 25.0 Italian was published 2026-03-09 and is the latest release at the time of training.
To complement the read-speech CV numbers above, we also benchmarked the YODAS-trained variants on an internal ~17-minute Italian conversational recording held back as a private real-world test set. The audio itself is not published, only the head-to-head numbers below. All runs on a single NVIDIA GB10 with CUDA: Whisper at q5_0 via whisper.cpp (CUDA backend), VibeVoice at q4_0 via vibevoice.cpp master (resident-KV + flash-attention).
| Model | Wall time | RTF (lower is faster) | 4+ word loop runs | Native diarization |
|---|---|---|---|---|
whisper-large-v3-it-yodas-only (q5_0) |
64.8 s | 0.065 ⚡ | 7 | , |
whisper-large-v3-turbo-it-yodas-only (q5_0) |
73.2 s | 0.073 | 62 | , |
vibevoice-asr-it-yodas-only (q4_0) |
619.9 s | 0.622 | 5 | ✅ |
A 4+ word loop run counts each position where the same lowercased token appears four times in a row, a crude but consistent proxy for hallucinated repetitions on tough audio. Lower is better.
Takeaways:
- For raw throughput,
whisper-large-v3-it-yodas-onlyis unbeatable on GPU (~15× realtime). - For loop-free transcripts and free speaker diarization,
vibevoice-asr-it-yodas-onlywins, at the cost of being only ~1.6× realtime. - Avoid
whisper-large-v3-turbo-it-yodas-onlyon this kind of audio, the smaller decoder degrades into long repetition runs (62 vs 7 for non-turbo), and is also slightly slower than non-turbo on GPU because of the extra fallback decode steps. Use it for read-speech where decoder length matters less.
LocalAI supports these models natively through the whisperx backend. Pick the variant that matches your accuracy / latency budget, drop the corresponding YAML into your LocalAI models/ directory, and transcribe via the OpenAI-compatible /v1/audio/transcriptions endpoint.
Quick hardware guide:
tiny/base, CPU-only edge devices, real-time on a Raspberry Pi.small/medium, most servers, great accuracy / speed tradeoff.large-v3-turbo,large-v3quality at roughly half the latency.large-v3, highest accuracy, when it matters most.
After setting up a model below, transcribe audio with:
curl http://localhost:8080/v1/audio/transcriptions \
-H "Content-Type: multipart/form-data" \
-F file="@audio.mp3" \
-F model="<the-name-from-the-yaml-below>"whisperx-tiny-it, Common Voice only
name: whisperx-tiny-it
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-tiny-it-ct2-int8
language: itwhisperx-tiny-it-multi, multi-dataset
name: whisperx-tiny-it-multi
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-tiny-it-multi-ct2-int8
language: itwhisperx-tiny-it-multi-yodas, multi + YODAS conversational data
name: whisperx-tiny-it-multi-yodas
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-tiny-it-multi-yodas-asr_only-nocv-ct2-int8
language: itwhisperx-base-it, Common Voice only
name: whisperx-base-it
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-base-it-ct2-int8
language: itwhisperx-base-it-multi, multi-dataset
name: whisperx-base-it-multi
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-base-it-multi-ct2-int8
language: itwhisperx-base-it-multi-yodas, multi + YODAS conversational data
name: whisperx-base-it-multi-yodas
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-base-it-multi-yodas-asr_only-nocv-ct2-int8
language: itwhisperx-small-it, Common Voice only
name: whisperx-small-it
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-small-it-ct2-int8
language: itwhisperx-small-it-multi, multi-dataset
name: whisperx-small-it-multi
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-small-it-multi-ct2-int8
language: itwhisperx-small-it-multi-yodas, multi + YODAS conversational data
name: whisperx-small-it-multi-yodas
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-small-it-multi-yodas-asr_only-nocv-ct2-int8
language: itwhisperx-medium-it, Common Voice only
name: whisperx-medium-it
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-medium-it-ct2-int8
language: itwhisperx-medium-it-multi, multi-dataset
name: whisperx-medium-it-multi
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-medium-it-multi-ct2-int8
language: itwhisperx-medium-it-multi-yodas, multi + YODAS conversational data
name: whisperx-medium-it-multi-yodas
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-medium-it-multi-yodas-asr_only-nocv-ct2-int8
language: itwhisperx-large-v3-it, Common Voice only
name: whisperx-large-v3-it
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-it-ct2-int8
language: itwhisperx-large-v3-it-multi, multi-dataset
name: whisperx-large-v3-it-multi
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-it-multi-ct2-int8
language: itwhisperx-large-v3-it-multi-yodas, multi + YODAS conversational data
name: whisperx-large-v3-it-multi-yodas
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-it-multi-yodas-asr_only-nocv-ct2-int8
language: itwhisperx-large-v3-it-yodas-only, pure YODAS (best on conversational)
name: whisperx-large-v3-it-yodas-only
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-it-yodas-only-ct2-int8
language: itwhisperx-large-v3-turbo-it, Common Voice only
name: whisperx-large-v3-turbo-it
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-turbo-it-ct2-int8
language: itwhisperx-large-v3-turbo-it-multi, multi-dataset
name: whisperx-large-v3-turbo-it-multi
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-turbo-it-multi-ct2-int8
language: itwhisperx-large-v3-turbo-it-yodas-only, pure YODAS (recommended for conversational)
name: whisperx-large-v3-turbo-it-yodas-only
backend: whisperx
known_usecases:
- transcript
parameters:
model: LocalAI-io/whisper-large-v3-turbo-it-yodas-only-ct2-int8
language: itfrom transformers import pipeline
pipe = pipeline(
"automatic-speech-recognition",
model="LocalAI-io/whisper-medium-it-multi",
)
result = pipe("audio.mp3", generate_kwargs={"language": "it", "task": "transcribe"})
print(result["text"])The -ct2-int8 repos are drop-in replacements for the upstream CT2 Whisper models:
from faster_whisper import WhisperModel
model = WhisperModel(
"LocalAI-io/whisper-medium-it-multi-ct2-int8",
device="cpu",
compute_type="int8",
)
segments, info = model.transcribe("audio.mp3", language="it")
for segment in segments:
print(f"[{segment.start:.1f}s - {segment.end:.1f}s] {segment.text}")Each -ggml repo contains four quantizations: ggml-model-f16.bin, ggml-model-q8_0.bin, ggml-model-q5_0.bin, ggml-model-q4_0.bin. Pick the one that fits your size / quality budget, q5_0 is the usual sweet spot, q8_0 is near-lossless, q4_0 is smallest.
huggingface-cli download LocalAI-io/whisper-medium-it-multi-ggml ggml-model-q5_0.bin --local-dir .
./whisper-cli -m ggml-model-q5_0.bin -f audio.wav -l itThese models also work with pywhispercpp and any other whisper.cpp-based runtime.
In addition to the Whisper variants above, we ship a VibeVoice ASR fine-tune trained on the same YODAS-only Italian mix (LoRA, ~10k steps on the GB10). VibeVoice is a different architecture (Qwen2-based with native diarization-aware decoding), useful when you want speaker-segmented output, or fewer hallucinated loops on tough conversational audio. On our private real-world conversational test set it produced fewer 4+ word repetition runs than Whisper large-v3-yodas (5 vs 7), with native speaker diarization in the output.
| Variant | Repo | Notes |
|---|---|---|
| Merged model | LocalAI-io/vibevoice-asr-it-yodas-only | Drop-in replacement for microsoft/VibeVoice-ASR. |
| LoRA adapter | LocalAI-io/vibevoice-asr-it-yodas-only-lora | Apply on top of the upstream base; ~tens of MB. |
| GGUF (vibevoice.cpp) | LocalAI-io/vibevoice-asr-it-yodas-only-ggml | q4_0, q5_0, q8_0 quantizations for vibevoice.cpp. |
from vibevoice.modular.modeling_vibevoice_asr import VibeVoiceASRForConditionalGeneration
from vibevoice.processor.vibevoice_asr_processor import VibeVoiceASRProcessor
import torch
processor = VibeVoiceASRProcessor.from_pretrained("LocalAI-io/vibevoice-asr-it-yodas-only")
model = VibeVoiceASRForConditionalGeneration.from_pretrained(
"LocalAI-io/vibevoice-asr-it-yodas-only",
dtype=torch.bfloat16,
attn_implementation="sdpa", # or "flash_attention_2" if installed
)Build vibevoice.cpp once (use -DVIBEVOICE_GGML_CUDA=ON for GPU), download a GGUF + the Qwen2.5 tokenizer GGUF, and run:
hf download LocalAI-io/vibevoice-asr-it-yodas-only-ggml \
vibevoice-asr-it-yodas-only-q5_0.gguf --local-dir models/
./build/bin/vibevoice-cli asr \
--model models/vibevoice-asr-it-yodas-only-q5_0.gguf \
--tokenizer models/tokenizer.gguf \
--audio audio.wav \
--max-new-tokens 8000 --verboseThe output is a JSON array of diarized segments (Start, End, Speaker, Content).
- Common Voice 25.0 Italian, crowdsourced read speech, the backbone of the CV-only and Multi training mixes.
- Multilingual LibriSpeech (MLS) Italian, audiobook readings, adds long-form narration to the Multi mix.
- VoxPopuli Italian, European Parliament recordings, adds parliamentary speech to the Multi mix.
- YODAS-Granary Italian, YouTube-sourced spontaneous speech, pseudo-labeled with
faster-whisper-large-v3and cleaned withQwen2.5-7B-Instruct. Provides the conversational coverage that the Multi mix alone is missing. Used in two slices:asr_only(~87k clips) for the Multi+YODAS variants, andasr_only+ast(200k cap) for the YODAS-only variants.
Models and code are released under the MIT license, consistent with the upstream OpenAI Whisper release.
If you use these models in academic work or want to cite this release, please use the following BibTeX entry:
@software{italian_asr_2026,
author = {Di Giacinto, Ettore and Palethorpe, Richard},
title = {{Italian ASR}: An open family of Italian speech-to-text models
(Whisper and VibeVoice fine-tunes) with a C++ inference engine
for VibeVoice},
year = {2026},
publisher = {LocalAI},
url = {https://github.com/mudler/italian-asr},
note = {Models on HuggingFace at \url{https://huggingface.co/LocalAI-io};
vibevoice.cpp engine at \url{https://github.com/mudler/vibevoice.cpp}}
}Plain-text:
Di Giacinto, E. and Palethorpe, R. (2026). Italian ASR: An open family of Italian speech-to-text models (Whisper and VibeVoice fine-tunes) with a C++ inference engine for VibeVoice. LocalAI. https://github.com/mudler/italian-asr
Built by Ettore Di Giacinto and Richard Palethorpe of the LocalAI team. If you find these models useful, please star LocalAI on GitHub, it genuinely helps the project.


