how to add my gpu for cuda

[MrScratchcat]

im sorry but i have no idea how to add my gpu to alpaca thanks for the help

[Jeffser]

Hi, what GPU do you use?

[MrScratchcat]

4060 ti

[Jeffser]

CUDA should work by default, Ollama might be actually defaulting to a different GPU (probably the iGPU from your CPU)

Could you run this command in a terminal? It should give you the UUID of your GPU

nvidia-smi -L

Then input that number inside the Alpaca preferences window under CUDA_VISIBLE_DEVICES

This should force Ollama to use your GPU!

[lepusgriseo]

I am wondering the same but unable to find where to find this parameter to change it to my GPU and not iGPU.

[Jeffser]

the option is now available in the instance preferences inside instance manager!

[Caalek]

Hi, I have set CUDA_VISIBLE_DEVICES to my GPU UUID from nvidia-smi -L, but Alpaca still uses the iGPU:

➜  ~ nvidia-smi -L
GPU 0: NVIDIA GeForce RTX 4060 Laptop GPU (UUID: GPU-e962dbf5-fd61-60a4-8301-beefeb53fb23)
➜  ~