Open
Description
This epic is a major tracker for all the backends additions that should be part of LocalAI v2 and ongoing efforts.
The objective is to release a v2 which deprecates old models which are now superseded, plus adding a new set. In order to achieve this my idea is to clean up the current state and start pinning dependencies for all the backends which requires specific environment settings (python-based ones).
Some backends will be deprecated as superseded, as such some repositories will be archived (TBD yet).
Backends:
- feat: add support for GPTQ-for-LLaMa #1055
- feat: Add support for LLaVA (text+images) and miniGPT4 #1054
- feat(exllama2): Add support to exllama2 #1053
- feat: tortoiseTTS #1016
- feat(transformer): transformer-based backend #1015
- [backend] llama-cpp, C++ gRPC backend #1154
- feat(alias): alias llama to llama-cpp, update docs #1448
- img2vid #1348
- feat(Petals): Add support to Petals #1262
- feat: split core and extra images with additional backends #1162
- feat(vllm): Initial vllm backend implementation #1026
- vall-e-x tts fails with ModuleNotFoundError: No module named 'utils' #1174
- typo in vall-e-x backend. #1370
- SDXL turbo #1380
- exllama(v2): fix exllamav1, add exllamav2 #1384
- feat(img2vid,txt2vid): Initial support for img2vid,txt2vid #1442
- coqui #1464
- feat: add transformers-musicgen backend #1387
- s4 mamba support #1588
- Localai v2 Image2Image not working #1392
- Support for Microsoft Phi-2 model #1438
- Vcoder #1529
- https://github.com/myshell-ai/OpenVoice #1639
- Support DeepSpeed FastGen #1538
- https://github.com/VikParuchuri/surya
- Backend: MLX #1638
- Model: whisperspeech #1636
- https://github.com/magic-research/magic-animate
- Support: moondream #1637
- feat(outlines): Add support pf outlines #1261
- Backend (lavie): Text to video #1640
- feat(deepSpare): Add support of DeepSpare #1263
- SAM support #915
- feature: support CTranslate2 #839
- feat: add support to ctransformers #1056
- feat: support chatglm #754
- feature: rust https://github.com/rustformers backend #939
- Support Qwen Model by Alibaba #1110
- Ollama support #1028
- https://github.com/chenxwh/insanely-fast-whisper
- https://twitter.com/ggerganov/status/1724898446770069983
- https://llm.mlc.ai/docs/deploy/python.html
- vit.cpp
- https://github.com/penghao-wu/vstar
- model pre-caching (huggingface/transformers) #1497
- model hotload (prepare it in ram before API is ready) #1539
- Tinygrad ( https://github.com/tinygrad/tinygrad/blob/master/examples/llama3.py )
- diffusers: add esrgan #917
- Diarization endpoint #1648
- Tokenization endpoint #1649
- backend(rwkv): python rwkv backend #1658
- Intel GPU support #1653
- feat(llama.cpp): Vulkan, Kompute, SYCL #1647
- Better Support for AMD and ROCM via docker containers. #1592
- https://github.com/apple/ml-mgie
- https://github.com/Zyphra/BlackMamba
- https://github.com/PKU-YuanGroup/MoE-LLaVA
- https://github.com/PKU-YuanGroup/Video-LLaVA
- https://github.com/LargeWorldModel/LWM
- https://github.com/Stability-AI/StableCascade/blob/master/inference%2Ftext_to_image.ipynb
- feat. add OpenVINO Model Server as a Backend #1722
- https://github.com/myshell-ai/MeloTTS
- [feature] TTS support request #1819
- https://github.com/roboflow/supervision
- https://github.com/Rudrabha/Wav2Lip
- https://github.com/jasonppy/VoiceCraft
- https://github.com/PKU-YuanGroup/Open-Sora-Plan
- feat: llamafile backend #2010
- Please support reranker API #2018
- feat(parler-tts): Add new backend #2027
- https://huggingface.co/nateraw/musicgen-songstarter-v0.2?s=09
- https://github.com/PABannier/bark.cpp
- feat(multimodal): Video understanding #2318
- https://github.com/systran/faster-whisper
- integrate whisperX #3375
- Please integrate chatTTS #2441
- https://github.com/fishaudio/fish-speech
- https://github.com/ictnlp/LLaMA-Omni
- Add support for cogvideo models #3628
- Moshi https://github.com/kyutai-labs/moshi/blob/main/moshi%2FREADME.md
- feat(multimodal): Audio understanding #3602
- https://github.com/baaivision/Emu3
- https://github.com/lucidrains/e2-tts-pytorch
- https://github.com/SWivid/F5-TTS
- https://github.com/homebrewltd/ichigo
- https://github.com/jy0205/Pyramid-Flow
- Support for SpiritLM #3966
- https://github.com/Tencent/HunyuanVideo
- tts : add OuteTTS support ggml-org/llama.cpp#10784