vLLM-Omni Model Support #808
Description
Motivation.
This section tracks the status of model integrations within the project, distinguishing between models that are actively under adaptation and those planned for future support. We hope the community can help us support these SOTA models together.
Proposed Change.
**P0:**🙋
Omni Pipeline
- Qwen/Qwen3-Omni update qwen-omni docs #559
- Qwen/Qwen2.5-Omni [Model]Add Qwen2.5-Omni model components #12
- MiMo-Audio [New Model]: MiMo-Audio from Xiaomi #151 [New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support #750
- HunyuanImage-3.0/Instruct/Instruct-Distil [New Model]: Add HunYuanImage3.0 #42 [New model] Support HY-Image3.0 DiT #794
- Bagel [New Model]: ByteDance-Seed/BAGEL-7B-MoT #203 Support Bagel Model #726
- zai-org/GLM-Image [Perf] GLM Image #792 [Fix] GLM Image #799 [Model][Rebase] Add GLM-Image Model and Partial Rebase to v0.14.0 (Support AR Offiline) #763 [Perf] GLM Image #920
- naver-hyperclovax/HyperCLOVAX-SEED-Omni-8B [New Model]: HyperCLOVAX-SEED-Omni-8B #743 [Model] Support HyperCLOVAX-SEED-Omni-8B #585
- inclusionAI/Ming-flash-omni-Preview [RFC]: Support Ming-Flash-Omni in vLLM-Omni #692
- stepfun-ai/Step-Audio-R1.1 [New Model]: Step-Audio-R1.1 #862
- Qwen/Qwen3-TTS [Model] Support Qwen3-TTS model series #895
- openbmb/MiniCPM-o-4_5 [New Model]: Omni model openbmb/MiniCPM-o-4_5 #1182
X2I
- Z-Image/Z-Image-Turbo [New Model]: Z-Image #99 [diffusion] z-image support #149
- Qwen-Image/Qwen-Image-2512 [Diffusion] Support Multi-image Generation and Add Web UI Demo for QwenImage #97
- Qwen-Image-Edit [New Model]: Qwen-Image-Edit #187 [Model] Add Qwen-Image-Edit #196
- Qwen-Image-Edit-2509 [Model] Support Qwen-Image-Edit 2509 ( multi-image input edit) #330
- Qwen-Image-Edit-2511 [Model] Support Qwen-Image-Edit 2511 #321
- Qwen-Image-Layered [New model] Support model qwen image layered #381
- Ovis-Image-7B [New Model]: Ovis-Image-7B #224 [Model] Ovis Image Model Addition #263
- FLUX.1-Kontext-dev [New Model]: FLUX.1-Kontext-dev #359 [model] support FLUX.1-Kontext-dev #561
- black-forest-labs/FLUX.1-schnell [New Model]: black-forest-labs/FLUX.1-schnell #574
- FLUX.2[dev] [New Model]: add FLUX.2 [dev] #153 [Model] Add Flux2 support #302
- FLUX.2-klein [Model] add flux2 klein #809
- Owen777/UltraFlux-v1 [New Model]: Owen777/UltraFlux-v1 #327 [model]Add UltraFlux-v1-image support #611
- LongCat-Image [New Model]: Add LongCat-Image #220 [Model] Add LongCat-Image support #291
- LongCat-Image-Edit [New Model]: LongCat-Image & LongCat-Image-Edit #338 [Model] Add LongCat-Image-Edit support #392
- stepfun-ai/NextStep-1.1 [New Model]: stepfun-ai/NextStep-1.1 #470 [Model]Add new nextstep_1(Diffusion) model #612
X2V
- Wan-AI/Wan2.2-T2V-A14B-Diffusers [Model] Add Wan2.2 text-to-video support #202
- Wan-AI/Wan2.2-I2V-A14B-Diffusers [New Model]: Wan-AI/Wan2.2-I2V-A14B-Diffusers #325
- Wan-AI/Wan2.2-TI2V-5B-Diffusers [Model] Add Wan2.2 I2V and TI2V pipeline support #329
- wan2.1-t2v-1.3b [New Model]: Wan2.1-T2V-1.3B #201 [Model] [WIP] Add Wan2.1 t2v support #244
- HunyuanVideo-1.5 [New Model]: HunyuanVideo-1.5 #228 [Model][WIP] Add hunyuan video 1.5 support #289
- Lightricks/LTX-2 [New Model]: Lightricks/LTX-2 #674 [Model] support Ltx2 text-to-video image-to-video #841
- OpenMOSS-Team/MOVA-720p [New Model]: OpenMOSS-Team/MOVA-720p #1092
- SkyReels-V3 [New Model]: Skywork / SkyReels #1093
X2S
- Stable Audio Open (stabilityai/stable-audio-open-1.0) [New Model]: stabilityai/stable-audio-open-1.0 #324 [Model] Add Stable Audio Open support for text-to-audio generation #331
- zai-org/GLM-TTS [New Model]: https://huggingface.co/zai-org/GLM-TTS #821 [Model] Add GLM-TTS text-to-speech model support #834
**P1:**🙋
Omni Pipeline
- LongCat-Flash-Omni [New Model]: LongCat-Flash-Omni #213
- Step-Audio2 [New Model]:Step Audio 2 #271 [WIP] [Model] Step-Audio2 #464
- Step-Audio-EditX [New Model]: Step Audio EditX #272
- MammothModa2-Preview [New Model]: bytedance-research/MammothModa2-Preview #314
- Fun-Audio-Chat [RFC][Model] Add Fun-Audio-Chat-8B Support #452
- Higgs audio v2 [New Model]: Higgs audio v2 #894
- Chatterbox TTS [New Model]: Chatterbox TTS #899
- VibeVoice-7B [New Model]: VibeVoice #184 [model] support VibeVoice ASR #999
X2I
- Stable Diffusion 3 [Model] Support stable diffusion3 #439
- OmniGen2 [New Model]: OmniGen2 #225 [Model] Support OmniGen2 #513
X2V
- IndexTeam/Index-anisora · Hugging Face [New Model]: Index-AniSora (Bilibili) #670
- tencent/HunyuanVideo-I2V [New Model]: add HunyuanVideo-I2V #383
X2S
- index-tts2 [New Model]: index-tts2 #229 [WIP] [Model] Index tts #334
- Fun-CosyVoice3-0.5B [New Model]: Fun-CosyVoice3-0.5B #315 [Model] Fun cosy voice3-0.5-b-2512 #498
Feedback Period.
No response
CC List.
@hsliuustc0106 @ZJY0516 @SamitHuang @wtomin @david6666666
Any Other Things.
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.