DS4 BUG with deepSeek v4 flash

[nahhasghira-maker]

I wanted to install DeepSeek V4 Flash. I installed DS4, but I found a bug: the DS4 GPU backend is not supported in llama.cpp. It only works on CPU.

[localai-bot]

Thanks for the report! Let me clarify what's happening, because there are a couple of things tangled together here.

DS4 isn't a llama.cpp backend. DeepSeek V4 Flash runs through ds4 (antirez/ds4), a separate single-model inference engine - not llama.cpp. So GPU support for it is independent of llama.cpp.

A DS4 GPU backend does exist and is published. There are two GPU builds:

• cuda13-ds4 - NVIDIA, CUDA 13
• metal-ds4 - Apple Silicon (Metal)

When you install the ds4 backend (or the DeepSeek V4 Flash model, which pulls it in), LocalAI auto-selects the variant based on detected hardware. It only resolves to a GPU build for:

• an NVIDIA GPU with CUDA 13, or
• Apple Silicon / Metal

Everything else falls back to the CPU build (cpu-ds4). In particular, DS4 currently has no ROCm/Vulkan/SYCL build and no CUDA 12 build, so on AMD/Intel GPUs - or an NVIDIA setup with an older CUDA toolkit/driver - you'll land on CPU-only. That's a coverage limitation of the DS4 engine, not a llama.cpp issue.

Could you share:

• Your GPU (vendor + model)
• Your CUDA toolkit / driver version (nvidia-smi if NVIDIA)
• Which backend variant got installed (visible in the backends list)

If you do have an NVIDIA CUDA-13-capable GPU, you can force the GPU build by installing the cuda13-ds4 backend explicitly. If it picked cpu-ds4 despite a compatible GPU, that's a detection bug we'll want to dig into.