Skip to content

ExLlamaSharp 1.4.0

Choose a tag to compare

View all tags
@Kortexio Kortexio released this 11 Sep 16:21
· 3 commits to main since this release
v1.4.0
545e7c2

ExLlamaSharp 1.4.0

Stable release promoting 1.4.0-beta Multi-GPU + the AdminLTE admin UI refresh.

[skip-announce]

Highlights

  • Real pipeline / tensor multi-GPU (Settings), combined VRAM fit badges, CUDA remap (strongest GPU → cuda:0)
  • Fail-fast prompt_too_long (no hang → client 408)
  • Admin console on AdminLTE 3 (Dashboard, Models, Chat, Settings, …)
  • OpenAI-compatible /v1, Models/Jobs/Keys, LoRA, speculative decoding, webhooks, tenants

Install

Download ExLlamaSharp-Setup-win-x64.exe below (Windows x64).
Default UI: http://127.0.0.1:14563

Notes

  • EXL3 models only (not GGUF)
  • Vision under multi-GPU remains text-only; OpenAI images/audio stay 501
Assets 3
Loading