Skip to content

GPU.md

Latest commit

 

History

History
40 lines (29 loc) · 2.16 KB

GPU.md

File metadata and controls

40 lines (29 loc) · 2.16 KB
Raw

GPU deployment

NVidia GPU EXPERIMENTAL support is available for analysis task in the worker process. This can significantly speed up processing of tracks.

We suggest 8GB VRAM on GPU, with less you can experience the NON BLOCKING OutOFMemory error (that are handled by switching to CPU). The PER_SONG_MODEL_RELOAD env variable, that by default is TRUE, help cleaning the memory by entirely reloading the model each time, on the other side it slow the analysis process.

GPU-accelerated clustering is also available through RAPIDS cuML. It can give a 10-30x speedup on clustering tasks.

Features:

  • GPU-accelerated KMeans, DBSCAN, and PCA using RAPIDS cuML
  • Automatic fallback to CPU if the GPU is unavailable or hits an error
  • Works with all existing clustering configurations and parameters
  • Compatible with NVIDIA GPUs on CUDA 12.8.1 or later (*)

(*) Older drivers are NOT supported by the published build, but you can try to build your own image as described in #265

To enable GPU clustering:

  1. Use the NVIDIA image (for example nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04)
  2. Set the value in your .env file, or in the Setup Wizard: 
    USE_GPU_CLUSTERING=true
  3. Make sure the NVIDIA Container Toolkit is installed on the host
  4. Use the GPU compose file deployment/docker-compose-nvidia.yaml. Worker-only GPU examples are kept in deployment/deprecated/

Performance Impact:

  • KMeans: 10-50x faster than CPU
  • DBSCAN: 5-100x faster than CPU
  • PCA: 10-40x faster than CPU
  • Overall clustering task: 10-30x speedup for typical workloads (5000 iterations)

Example: A clustering task that takes 2-4 hours on CPU may finish in 5-15 minutes on GPU.

Notes:

  • GMM and Spectral clustering stay on CPU, there is no cuML implementation for them in this build
  • GPU clustering is disabled by default (USE_GPU_CLUSTERING=false)
  • The GPU is also used by the audio analysis models (ONNX inference)
  • The index build and the similarity queries are not GPU accelerated; they are IO bound rather than compute bound, see ALGORITHM