If you find this project useful, please give it a star! Your support is appreciated and helps keep the project growing. 🌟
This guide walks you through installing NVIDIA CUDA Toolkit 11.8, cuDNN, and TensorRT on Windows, including setting up Python packages like Cupy and TensorRT. It ensures proper system configuration for CUDA development, with steps for setting environment variables and verifying installation via cmd.exe
First, download the CUDA Toolkit 11.8 from the official NVIDIA website:
👉 Nvidia CUDA Toolkit 11.8 - DOWNLOAD HERE
- After downloading, open the installer (
.exe) and follow the instructions provided by the installer. - Make sure to select the following components during installation:
- CUDA Toolkit
- CUDA Samples
- CUDA Documentation (optional)
- After the installation completes, open the
cmd.exeterminal and run the following command to ensure that CUDA has been installed correctly:nvcc --version
This will display the installed CUDA version.
Run the following command in your terminal to install Cupy:
pip install cupy-cuda11x
Download cuDNN (CUDA Deep Neural Network library) from the NVIDIA website:
👉 Download CUDNN. (Requires an NVIDIA account – it's free).
Open the .zip cuDNN file and move all the folders/files to the location where the CUDA Toolkit is installed on your machine, typically:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
Download TensorRT 8.6 GA.
Open the .zip TensorRT file and move all the folders/files to the CUDA Toolkit folder, typically located at:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
Once all the files are copied, run the following command to install TensorRT for Python:
pip install "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\python\tensorrt-8.6.1-cp311-none-win_amd64.whl"
🚨 Note: If this step doesn’t work, double-check that the .whl file matches your Python version (e.g., cp311 is for Python 3.11). Just locate the correct .whl file in the python folder and replace the path accordingly.
Add the following paths to your environment variables:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\lib
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\libnvvp
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin
Once you have CUDA 11.8 installed and cuDNN properly configured, you need to set up your environment via cmd.exe to ensure that the system uses the correct version of CUDA (especially if multiple CUDA versions are installed).
You need to add the CUDA 11.8 binaries to the environment variables in the current cmd.exe session.
Open cmd.exe and run the following commands:
set PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin;%PATH%
set PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\libnvvp;%PATH%
set PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\extras\CUPTI\lib64;%PATH%
These commands add the CUDA 11.8 binary, lib, and CUPTI paths to your system's current session. Adjust the paths as necessary depending on your installation directory.
- Verify the CUDA Version After setting the paths, you can verify that your system is using CUDA 11.8 by running:
nvcc --version
This should display the details of CUDA 11.8. If it shows a different version, check the paths and ensure the proper version is set.
-
Set the Environment Variables for a Persistent Session If you want to ensure CUDA 11.8 is used every time you open
cmd.exe, you can add these paths to your system environment variables permanently: -
Open
Control Panel->System->Advanced System Settings. Click onEnvironment Variables. UnderSystem variables, selectPathand clickEdit. Add the following entries at the top of the list:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\libnvvp
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\extras\CUPTI\lib64
This ensures that CUDA 11.8 is prioritized when running CUDA applications, even on systems with multiple CUDA versions.
- Set CUDA Environment Variables for cuDNN
If you're using cuDNN, ensure the
cudnn64_8.dllis also in your system path:
set PATH=C:\tools\cuda\bin;%PATH%
This should properly set up CUDA 11.8 to be used for your projects via cmd.exe.
import torch
print(torch.cuda.is_available()) # This will return True if CUDA is available
print(torch.version.cuda) # This will print the CUDA version being used
print(torch.cuda.get_device_name(0)) # This will print the name of the GPU, e.g., 'NVIDIA GeForce RTX GPU Model'
run the get_device.py to see if you installed it correctly
- run the
cuda-requirements.batafter you get done with installion of nvidia.
@echo off
:: Batch script to install Python packages for CUDA 11.8 environment
echo MAKE SURE TO HAVE THE WHL DOWNLOADED BEFORE YOU CONTINUE!!!
pause
echo Click the link to download the WHL: press ctrl then left click with mouse
echo https://github.com/cupy/cupy/releases/download/v13.4.1/cupy_cuda11x-13.4.1-cp311-cp311-win_amd64.whl
pause
echo Installing CuPy from WHL...
pip install https://github.com/cupy/cupy/releases/download/v13.4.1/cupy_cuda11x-13.4.1-cp311-cp311-win_amd64.whl
echo Press enter to continue with the rest of the dependency installs
pause
echo Installing ONNX Runtime with GPU support...
pip install onnxruntime-gpu==1.19.2
echo Press enter to continue with the rest of the dependency installs
pause
echo Installing NVIDIA PyIndex...
pip install nvidia-pyindex
echo Press enter to continue with the rest of the dependency installs
pause
echo Installing cuDNN for CUDA 11.8...
pip install nvidia-cudnn-cu11==8.6.0.163
echo Press enter to continue with the rest of the dependency installs
pause
echo Installing TensorRT for CUDA 11.8...
pip install nvidia-tensorrt==8.6.1
echo Press enter to continue with the rest of the dependency installs
pause
echo Installing NumPy...
pip install numpy
echo Press enter to continue with the rest of the dependency installs
pause
echo Installing cupy-cuda11x...
pip install cupy-cuda11x
echo Press enter to continue with the rest of the dependency installs
pause
echo All packages installed successfully!
pause