vLLM Cuda 13

[okcodemaybe]

Hi all,

Love LocalAI!
Was wondering if there were plans in adding CUDA 13 vLLM support?
I see its not listed as a supported backend on the wiki and when I manually try to load it from host-path backend directory it does not work.
Thanks again!

[itxashancode]

vLLM with CUDA 13 support isn't currently listed in the supported backends, and the host-path loading approach you tried suggests it's not yet integrated. The team would need to verify CUDA 13 compatibility and update the build pipeline. If you're comfortable with it, testing a custom build and reporting back would help prioritize this. Otherwise, keep an eye on the repo for updates - CUDA 13 support could be added in a future release if there's enough demand.