SUPPORTED_MODELS.md

OpenVINO™ GenAI: Supported Models

Large language models

Architecture	Models	Example HuggingFace Models
ChatGLMModel	ChatGLM	THUDM/chatglm3-6b
GemmaForCausalLM	Gemma	google/gemma-2b-it google/gemma-7b-it
GPTNeoXForCausalLM	Dolly	databricks/dolly-v2-3b
	RedPajama	ikala/redpajama-3b-chat
LlamaForCausalLM	Llama 3	meta-llama/Meta-Llama-3-8B meta-llama/Meta-Llama-3-8B-Instruct meta-llama/Meta-Llama-3-70B meta-llama/Meta-Llama-3-70B-Instruct
	Llama 2	meta-llama/Llama-2-13b-chat-hf meta-llama/Llama-2-13b-hf meta-llama/Llama-2-7b-chat-hf meta-llama/Llama-2-7b-hf meta-llama/Llama-2-70b-chat-hf meta-llama/Llama-2-70b-hf microsoft/Llama2-7b-WhoIsHarryPotter
	OpenLLaMA	openlm-research/open_llama_13b openlm-research/open_llama_3b openlm-research/open_llama_3b_v2 openlm-research/open_llama_7b openlm-research/open_llama_7b_v2
	TinyLlama	TinyLlama/TinyLlama-1.1B-Chat-v1.0
MistralForCausalLM	Mistral	mistralai/Mistral-7B-v0.1
	Notus	argilla/notus-7b-v1
	Zephyr	HuggingFaceH4/zephyr-7b-beta
PhiForCausalLM	Phi	microsoft/phi-2 microsoft/phi-1_5
QWenLMHeadModel	Qwen	Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat-Int4 Qwen/Qwen1.5-7B-Chat Qwen/Qwen1.5-7B-Chat-GPTQ-Int4

Note

LoRA adapters are supported.

The pipeline can work with other similar topologies produced by optimum-intel with the same model signature. The model is required to have the following inputs after the conversion:

1. input_ids contains the tokens.
2. attention_mask is filled with 1.
3. beam_idx selects beams.
4. position_ids (optional) encodes a position of currently generating token in the sequence and a single logits output.

Note

Models should belong to the same family and have the same tokenizers.

Image generation models

Architecture	Text 2 image	Image 2 image	Inpainting	LoRA support	Example HuggingFace Models
Latent Consistency Model	Supported	Supported	Supported	Supported	SimianLuo/LCM_Dreamshaper_v7
Stable Diffusion	Supported	Supported	Supported	Supported	CompVis/stable-diffusion-v1-1 CompVis/stable-diffusion-v1-2 CompVis/stable-diffusion-v1-3 CompVis/stable-diffusion-v1-4 jcplus/stable-diffusion-v1-5 stable-diffusion-v1-5/stable-diffusion-v1-5 botp/stable-diffusion-v1-5 dreamlike-art/dreamlike-anime-1.0 stabilityai/stable-diffusion-2 stabilityai/stable-diffusion-2-base stabilityai/stable-diffusion-2-1 bguisard/stable-diffusion-nano-2-1 justinpinkney/pokemon-stable-diffusion stablediffusionapi/architecture-tuned-model IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-EN-v0.1 ZeroCool94/stable-diffusion-v1-5 pcuenq/stable-diffusion-v1-4 rinna/japanese-stable-diffusion benjamin-paine/stable-diffusion-v1-5 philschmid/stable-diffusion-v1-4-endpoints naclbit/trinart_stable_diffusion_v2 Fictiverse/Stable_Diffusion_PaperCut_Model
Stable Diffusion Inpainting	Not applicable	Not applicable	Supported	Supported	stabilityai/stable-diffusion-2-inpainting stable-diffusion-v1-5/stable-diffusion-inpainting botp/stable-diffusion-v1-5-inpainting parlance/dreamlike-diffusion-1.0-inpainting
Stable Diffusion XL	Supported	Supported	Supported	Supported	stabilityai/stable-diffusion-xl-base-0.9 stabilityai/stable-diffusion-xl-base-1.0 stabilityai/sdxl-turbo cagliostrolab/animagine-xl-4.0
Stable Diffusion XL Inpainting	Not applicable	Not applicable	Supported	Supported	diffusers/stable-diffusion-xl-1.0-inpainting-0.1
Stable Diffusion 3	Supported	Supported	Supported	Partially supported	stabilityai/stable-diffusion-3-medium-diffusers stabilityai/stable-diffusion-3.5-medium stabilityai/stable-diffusion-3.5-large stabilityai/stable-diffusion-3.5-large-turbo
Flux	Supported	Supported	Supported	Partially Supported	black-forest-labs/FLUX.1-schnell Freepik/flux.1-lite-8B-alpha black-forest-labs/FLUX.1-dev shuttleai/shuttle-3-diffusion shuttleai/shuttle-3.1-aesthetic shuttleai/shuttle-jaguar Shakker-Labs/AWPortrait-FL black-forest-labs/FLUX.1-Fill-dev

Visual language models

Architecture	Models	LoRA support	Example HuggingFace Models	Notes
InternVL2	InternVL2	Not supported	OpenGVLab/InternVL2-1B OpenGVLab/InternVL2-2B OpenGVLab/InternVL2-4B OpenGVLab/InternVL2-8B OpenGVLab/InternVL2_5-1B OpenGVLab/InternVL2_5-2B OpenGVLab/InternVL2_5-4B OpenGVLab/InternVL2_5-8B
LLaVA	LLaVA-v1.5	Not supported	llava-hf/llava-1.5-7b-hf
LLaVA-NeXT	LLaVa-v1.6	Not supported	llava-hf/llava-v1.6-mistral-7b-hf llava-hf/llava-v1.6-vicuna-7b-hf llava-hf/llama3-llava-next-8b-hf
MiniCPMV	MiniCPM-V-2_6	Not supported	openbmb/MiniCPM-V-2_6
Phi3VForCausalLM	phi3_v	Not supported	microsoft/Phi-3-vision-128k-instruct microsoft/Phi-3.5-vision-instruct	These models' configs aren't consistent. It's required to override the default eos_token_id with the one from a tokenizer: generation_config.set_eos_token_id(pipe.get_tokenizer().get_eos_token_id()).
Qwen2-VL	Qwen2-VL	Not supported	Qwen/Qwen2-VL-2B-Instruct Qwen/Qwen2-VL-7B-Instruct

Whisper models

Architecture	Models	LoRA support	Example HuggingFace Models
WhisperForConditionalGeneration	Whisper	Not supported	openai/whisper-tiny openai/whisper-tiny.en openai/whisper-base openai/whisper-base.en openai/whisper-small openai/whisper-small.en openai/whisper-medium openai/whisper-medium.en openai/whisper-large-v3
	Distil-Whisper	Not supported	distil-whisper/distil-small.en distil-whisper/distil-medium.en distil-whisper/distil-large-v3

Text embeddings models

Architecture	LoRA support	Example HuggingFace Models
BertModel	Not supported	BAAI/bge-small-en-v1.5 BAAI/bge-base-en-v1.5 BAAI/bge-large-en-v1.5 sentence-transformers/all-MiniLM-L12-v2 mixedbread-ai/mxbai-embed-large-v1 mixedbread-ai/mxbai-embed-xsmall-v1 WhereIsAI/UAE-Large-V1
MPNetForMaskedLM	Not supported	sentence-transformers/all-mpnet-base-v2 sentence-transformers/multi-qa-mpnet-base-dot-v1
RobertaForMaskedLM	Not supported	sentence-transformers/all-distilroberta-v1
XLMRobertaModel	Not supported	mixedbread-ai/deepset-mxbai-embed-de-large-v1 intfloat/multilingual-e5-large-instruct intfloat/multilingual-e5-large

Some models may require access request submission on the Hugging Face page to be downloaded.

Speech generation models

Architecture	Models	LoRA support	Example HuggingFace Models
SpeechT5ForTextToSpeech	SpeechT5 TTS	Not supported	microsoft/speecht5_tts

If https://huggingface.co/ is down, the conversion step won't be able to download the models.