Replies: 1 comment
-
And i just found out that Docker Desktop includes everything needed of the "nvidia container toolkit" So using Docker Desktop, and latest nvidia drivers, we users do not need to concern ourselves at all with any extra steps to fully utilise our cuda nvidia gpus. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
This has taken me a week!!!!!! to figure out. And the solution is a slap in the face.
https://github.com/n8n-io/self-hosted-ai-starter-kit
. . . tells nvidia gpu users to follow Ollama Docker Instructions . . .
After going down the rabbit hole of Ollama Docker -> nVidia Contain Toolkit -> NVIDIA GPU driver for your Linux distribution . . .
discovering that method is well out of date - we need windows only drivers . . . finding nvidia cuda drivers with WSL 2 GPU Paravirtualization support in the dev areas . . . And totally ruining my pc again, and again, again . . .
i eventually found this
https://developer.nvidia.com/cuda/wsl
which takes you to the regular nvidia drivers AS THEY INCLUDE CUDA-FOR-WSL2
i dont know whether to laugh or scream. i've been down rabbit holes inside rabbit holes. My brain is fried.
i strongly suggest the instructions are updated. Maybe not say anything about nvidia vs amd gpu's?! or just a sentence saying you're ok, if youre drivers are from 2025 !! (or whenever this changed).
I also found this
https://docs.docker.com/desktop/features/gpu/
which has a wonderful command to test your setup
docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
my fresh install of everything passed. So once ive recovered i can continue...
anyone know about this? an extra sentence or two might save others HOURS of headaches.
Thanks.
Beta Was this translation helpful? Give feedback.
All reactions