Immich + NVIDIA GPU for Video Transcoding and ML

[adityataps]

NVIDIA GPU setup for Immich transcoding and ML

This is a quick-and-dirty guide to get your NVIDIA GPU working with the Immich LXC, which can be used for video transcoding and ML features like facial recognition.

Warning

I'm just a hobbyist, and not in any way a developer on Immich. Please make backups of your containers and follow this guide at your own risk.

---

Prerequisites

• LXC (Proxmox) running Immich >=v2.0.1 (Community Scripts).
  • OS should be Debian 13 (Trixie).
  • This guide is tested with an unprivileged LXC.
• Working NVIDIA driver on host (nvidia-smi).
  • Check this tutorial on setting up the GPU on the host.

NVIDIA GPU LXC passthrough

Important

If you have existing data in your Immich LXC, back up your container!

1. In /etc/pve/lxc/<CTID>.conf, add these lines:

   lxc.cgroup2.devices.allow: c 195:* rwm
   lxc.cgroup2.devices.allow: c 226:* rwm
   lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
   lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
   lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
   lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
   lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
   

   Reboot the container using pct reboot <CTID>.

2. Locate the NVIDIA Linux driver that corresponds with the NVIDIA driver version on the host (as seen in nvidia-smi; for me, it was 550.163.01). Copy the URL of the .run file of the corresponding driver.

   

3. In the LXC, enter these commands:

   wget https://download.nvidia.com/XFree86/Linux-x86_64/550.163.01/NVIDIA-Linux-x86_64-550.163.01.run # replace with your .run url
   chmod +x NVIDIA-Linux-x86_64-550.163.01.run # replace with your .run file
   ./NVIDIA-Linux-x86_64-550.163.01.run --no-kernel-module # --no-kernel-module is important!

   Reboot the container. After rebooting, you should now be able to run nvidia-smi from within the container:

   root@immich-demo:~# nvidia-smi
   Tue Oct  7 23:11:28 2025       
   +-----------------------------------------------------------------------------------------+
   | NVIDIA-SMI 550.163.01             Driver Version: 550.163.01     CUDA Version: 12.4     |
   |-----------------------------------------+------------------------+----------------------+
   | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
   | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
   |                                         |                        |               MIG M. |
   |=========================================+========================+======================|
   |   0  NVIDIA GeForce RTX 3060        Off |   00000000:2B:00.0 Off |                  N/A |
   |  0%   40C    P8             14W /  170W |       2MiB /  12288MiB |      0%      Default |
   |                                         |                        |                  N/A |
   +-----------------------------------------+------------------------+----------------------+
                                                                                            
   +-----------------------------------------------------------------------------------------+
   | Processes:                                                                              |
   |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
   |        ID   ID                                                               Usage      |
   |=========================================================================================|
   |  No running processes found                                                             |
   +-----------------------------------------------------------------------------------------+
   root@immich-demo:~# 
   

Transcoding configuration

1. After following the previous steps for GPU passthrough, go to the Immich webapp and navigate to Administration > Video Transcoding Settings. Set "Acceleration API" to NVENC and save settings.

   

CUDA configuration for ML features

Note

For these steps I will be using CUDA 12.4. For your installation please refer to your
nvidia-smi output to determine which version of CUDA and related packages to install.

1. In the LXC terminal, open the Immich ML logs:

   tail -f --lines 100 /var/log/immich/ml.log

2. In the Immich webapp, upload a new image. You should start to see some logs in the LXC terminal. (I enabled OpenVINO by mistake when setting up the demo LXC, so your logs might be different.)

   You might see some logs like this:

   [10/07/25 23:11:21] INFO     Application startup complete.                      
   [10/07/25 23:30:47] INFO     Downloading detection model 'buffalo_l'. This may  
                                take a while.                                      
   Fetching 4 files: 100%|██████████| 4/4 [00:06<00:00,  1.69s/it]
   [10/07/25 23:30:57] INFO     Loading detection model 'buffalo_l' to memory      
   [10/07/25 23:30:58] INFO     Setting execution providers to                     
                                ['OpenVINOExecutionProvider',                      
                                'CPUExecutionProvider'], in descending order of    
                                preference                                         
   75 warnings generated.
   75 warnings generated.
   75 warnings generated.
   75 warnings generated.
   75 warnings generated.
   75 warnings generated.
   75 warnings generated.
   [10/07/25 23:30:59] INFO     Downloading visual model 'ViT-B-32__openai'. This  
                                may take a while.                                  
   75 warnings generated.
   Fetching 11 files:   0%|          | 0/11 [00:00<?, ?it/s]77 warnings generated.
   75 warnings generated.
   Fetching 11 files:   9%|▉         | 1/11 [00:00<00:01,  8.34it/s]77 warnings generated.
   75 warnings generated.
   77 warnings generated.
   75 warnings generated.
   77 warnings generated.
   2025-10-07 23:31:00.014956587 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running OpenVINO-EP-subgraph_1 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0' Status Message: /onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:243 void onnxruntime::openvino_ep::OVInferRequest::WaitRequest() [OpenVINO-EP]  Wait Model Failed: Exception from src/inference/src/cpp/infer_request.cpp:245:
   Check 'data_rank.is_dynamic() || filters_rank.is_dynamic() || data_shape[1].compatible(filters_shape[1])' failed at src/core/shape_inference/include/convolution_shape_inference_util.hpp:314:
   While validating node 'opset1::Convolution Convolution_6646 () -> ()' with friendly_name 'Convolution_6646':
   Data batch channel count (0) does not match filter input channel count (56).
   
   
   
   75 warnings generated.
   75 warnings generated.
   [10/07/25 23:31:00] ERROR    Exception in ASGI application                      
                                                                                   
                                ╭─────── Traceback (most recent call last) ───────╮
                                │ /opt/immich/app/machine-learning/immich_ml/main │
                                │ .py:177 in predict                              │
                                │                                                 │
                                │   174 │   │   inputs = text                     │
                                │   175 │   else:                                 │
                                │   176 │   │   raise HTTPException(400, "Either  │
                                │ ❱ 177 │   response = await run_inference(inputs │
                                │   178 │   return ORJSONResponse(response)       │
                                │   179                                           │
                                │   180                                           │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/main │
                                │ .py:200 in run_inference                        │
                                │                                                 │
                                │   197 │   │   response[entry["task"]] = output  │
                                │   198 │                                         │
                                │   199 │   without_deps, with_deps = entries     │
                                │ ❱ 200 │   await asyncio.gather(*[_run_inference │
                                │   201 │   if with_deps:                         │
                                │   202 │   │   await asyncio.gather(*[_run_infer │
                                │   203 │   if isinstance(payload, Image):        │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/main │
                                │ .py:195 in _run_inference                       │
                                │                                                 │
                                │   192 │   │   │   │   message = f"Task {entry[' │
                                │       output of {dep}"                          │
                                │   193 │   │   │   │   raise HTTPException(400,  │
                                │   194 │   │   model = await load(model)         │
                                │ ❱ 195 │   │   output = await run(model.predict, │
                                │   196 │   │   outputs[model.identity] = output  │
                                │   197 │   │   response[entry["task"]] = output  │
                                │   198                                           │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/main │
                                │ .py:213 in run                                  │
                                │                                                 │
                                │   210 │   if thread_pool is None:               │
                                │   211 │   │   return func(*args, **kwargs)      │
                                │   212 │   partial_func = partial(func, *args, * │
                                │ ❱ 213 │   return await asyncio.get_running_loop │
                                │   214                                           │
                                │   215                                           │
                                │   216 async def load(model: InferenceModel) ->  │
                                │                                                 │
                                │ /opt/immich/.local/share/uv/python/cpython-3.11 │
                                │ .13-linux-x86_64-gnu/lib/python3.11/concurrent/ │
                                │ futures/thread.py:58 in run                     │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/mode │
                                │ ls/base.py:61 in predict                        │
                                │                                                 │
                                │    58 │   │   self.load()                       │
                                │    59 │   │   if model_kwargs:                  │
                                │    60 │   │   │   self.configure(**model_kwargs │
                                │ ❱  61 │   │   return self._predict(*inputs, **m │
                                │    62 │                                         │
                                │    63 │   @abstractmethod                       │
                                │    64 │   def _predict(self, *inputs: Any, **mo │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/mode │
                                │ ls/facial_recognition/detection.py:30 in        │
                                │ _predict                                        │
                                │                                                 │
                                │   27 │   def _predict(self, inputs: NDArray[np. │
                                │      FaceDetectionOutput:                       │
                                │   28 │   │   inputs = decode_cv2(inputs)        │
                                │   29 │   │                                      │
                                │ ❱ 30 │   │   bboxes, landmarks = self._detect(i │
                                │   31 │   │   return {                           │
                                │   32 │   │   │   "boxes": bboxes[:, :4].round() │
                                │   33 │   │   │   "scores": bboxes[:, 4],        │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/mode │
                                │ ls/facial_recognition/detection.py:38 in        │
                                │ _detect                                         │
                                │                                                 │
                                │   35 │   │   }                                  │
                                │   36 │                                          │
                                │   37 │   def _detect(self, inputs: NDArray[np.u │
                                │      NDArray[np.float32]]:                      │
                                │ ❱ 38 │   │   return self.model.detect(inputs)   │
                                │   39 │                                          │
                                │   40 │   def configure(self, **kwargs: Any) ->  │
                                │   41 │   │   self.model.det_thresh = kwargs.pop │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/ml-venv/lib/py │
                                │ thon3.11/site-packages/insightface/model_zoo/re │
                                │ tinaface.py:224 in detect                       │
                                │                                                 │
                                │   221 │   │   det_img = np.zeros( (input_size[1 │
                                │   222 │   │   det_img[:new_height, :new_width,  │
                                │   223 │   │                                     │
                                │ ❱ 224 │   │   scores_list, bboxes_list, kpss_li │
                                │   225 │   │                                     │
                                │   226 │   │   scores = np.vstack(scores_list)   │
                                │   227 │   │   scores_ravel = scores.ravel()     │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/ml-venv/lib/py │
                                │ thon3.11/site-packages/insightface/model_zoo/re │
                                │ tinaface.py:152 in forward                      │
                                │                                                 │
                                │   149 │   │   kpss_list = []                    │
                                │   150 │   │   input_size = tuple(img.shape[0:2] │
                                │   151 │   │   blob = cv2.dnn.blobFromImage(img, │
                                │       (self.input_mean, self.input_mean, self.i │
                                │ ❱ 152 │   │   net_outs = self.session.run(self. │
                                │   153 │   │                                     │
                                │   154 │   │   input_height = blob.shape[2]      │
                                │   155 │   │   input_width = blob.shape[3]       │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/immich_ml/sess │
                                │ ions/ort.py:49 in run                           │
                                │                                                 │
                                │    46 │   │   input_feed: dict[str, NDArray[np. │
                                │    47 │   │   run_options: Any = None,          │
                                │    48 │   ) -> list[NDArray[np.float32]]:       │
                                │ ❱  49 │   │   outputs: list[NDArray[np.float32] │
                                │       run_options)                              │
                                │    50 │   │   return outputs                    │
                                │    51 │                                         │
                                │    52 │   @property                             │
                                │                                                 │
                                │ /opt/immich/app/machine-learning/ml-venv/lib/py │
                                │ thon3.11/site-packages/onnxruntime/capi/onnxrun │
                                │ time_inference_collection.py:220 in run         │
                                │                                                 │
                                │    217 │   │   if not output_names:             │
                                │    218 │   │   │   output_names = [output.name  │
                                │    219 │   │   try:                             │
                                │ ❱  220 │   │   │   return self._sess.run(output │
                                │    221 │   │   except C.EPFail as err:          │
                                │    222 │   │   │   if self._enable_fallback:    │
                                │    223 │   │   │   │   print(f"EP Error: {err!s │
                                ╰─────────────────────────────────────────────────╯
                                Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero     
                                status code returned while running                 
                                OpenVINO-EP-subgraph_1 node.                       
                                Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgrap
                                h_1_0' Status Message:                             
                                /onnxruntime/onnxruntime/core/providers/openvino/ov
                                _interface.cc:243 void                             
                                onnxruntime::openvino_ep::OVInferRequest::WaitReque
                                st() [OpenVINO-EP]  Wait Model Failed: Exception   
                                from src/inference/src/cpp/infer_request.cpp:245:  
                                Check 'data_rank.is_dynamic() ||                   
                                filters_rank.is_dynamic() ||                       
                                data_shape[1].compatible(filters_shape[1])' failed 
                                at                                                 
                                src/core/shape_inference/include/convolution_shape_
                                inference_util.hpp:314:                            
                                While validating node 'opset1::Convolution         
                                Convolution_6646 () -> ()' with friendly_name      
                                'Convolution_6646':                                
                                Data batch channel count (0) does not match filter 
                                input channel count (56).                             
   

   The key line is here:

   [10/07/25 23:30:58] INFO     Setting execution providers to                     
                                ['OpenVINOExecutionProvider',                      
                                'CPUExecutionProvider'], in descending order of    
                                preference                                         
   

   We need the execution provider to be CUDAExecutionProvider, and to do that, we need onnxruntime to be able to detect our NVIDIA GPU.

3. Stop the Immich services:

   systemctl stop immich-ml immich-web

4. Activate the ml-venv uv virtual environment and use uv pip list to find the currently installed onnx and
   cuda/cudnn runtime DLLs:

   cd /opt/immich/app/machine-learning/
   source ml-venv/bin/activate
   uv pip list | grep 'onnx\|cuda\|cudnn'

   This is my output:

   Using Python 3.11.13 environment at: ml-venv
   onnx 1.16.0
   onnxruntime-openvino 1.18.0

   If you don't see onnxruntime-gpu or any cuda packages, install the onnxruntime-gpu package with cuda and cudnn extras (while the ml-venv virtualenv is still active):

   uv pip install onnxruntime-gpu[cuda,cudnn]

   You should now be able to see the installed onnxruntime-gpu package along with the relevant cuda and cudnn runtime DLLs:

   (immich-ml) root@immich:~# uv pip list | grep 'onnx\|cuda\|cudnn'
   Using Python 3.11.13 environment at: /opt/immich/app/machine-learning/ml-venv
   nvidia-cuda-nvrtc-cu12   12.9.86
   nvidia-cuda-runtime-cu12 12.9.79
   nvidia-cudnn-cu12        9.13.1.26
   onnx                     1.16.0
   onnxruntime-gpu          1.23.0
   onnxruntime-openvino     1.18.0

Tip

Refer to the onnxruntime compatability matrix for the compatible versions of cuda and cudnn.

6. Install CUDA Toolkit for your expected CUDA version, as seen on nvidia-smi. For me, this was 12.4. These instructions can also be found on the NVIDIA website. Installing the keyring:

   wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
   sudo dpkg -i cuda-keyring_1.1-1_all.deb
   sudo add-apt-repository contrib
   sudo apt-get update
   sudo apt-get -y install cuda-toolkit-12-4

   If the above doesn't work (add-apt-repository not found, Depends: libtinfo5 but it is not installable, etc.), try installing the libraries individually.

   sudo apt-get update
   sudo apt-get install \
    cuda-toolkit-12-4-config-common \
    cuda-nvcc-12-4 \
    cuda-cudart-dev-12-4 \
    cuda-libraries-dev-12-4

   After installing verify your CUDA Toolkit installation by updating the PATH and LD_LIBRARY_PATH variables and verifying your output from nvcc --version. More post-installation instructions can be found here.

   export PATH=${PATH}:/usr/local/cuda-12.4/bin
   export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-12.4/lib64
   nvcc --version

7. To update PATH and LD_LIBRARY_PATH to point to cuda and cudnn in the ML systemd service, add these lines to /etc/systemd/system/immich-ml.service, under the [Service] block:

   Environment="PATH=/usr/local/cuda-12.4/bin"
   Environment="LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/curand/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cufft/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/nvjitlink/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cuda_runtime/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cudnn/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cuda_nvrtc/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cublas/lib"
   

Note

Your paths may look a little different depending on what CUDA version you're running. To figure out your
LD_LIBRARY_PATH value, you can run

{ echo "/usr/local/cuda-12.4/lib64"; find /opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia -type d -name "lib"; } | paste -sd:

8. Run systemctl daemon-reload and systemctl start immich-ml immich-web, or reboot the container. Open the logs again (tail -f --lines 100 /var/log/immich/ml.log) and upload an image again from the webapp. You should now see logs like this:

   [10/08/25 00:42:52] INFO     Starting gunicorn 23.0.0                           
   [10/08/25 00:42:52] INFO     Listening at: http://[::]:3003 (1896)              
   [10/08/25 00:42:52] INFO     Using worker: immich_ml.config.CustomUvicornWorker 
   [10/08/25 00:42:52] INFO     Booting worker with pid: 1897                      
   [10/08/25 00:42:53] INFO     Started server process [1897]                      
   [10/08/25 00:42:53] INFO     Waiting for application startup.                   
   [10/08/25 00:42:53] INFO     Created in-memory cache with unloading after 300s  
                                of inactivity.                                     
   [10/08/25 00:42:53] INFO     Initialized request thread pool with 4 threads.    
   [10/08/25 00:42:53] INFO     Application startup complete.                      
   [10/08/25 00:43:32] INFO     Loading detection model 'buffalo_l' to memory      
   [10/08/25 00:43:32] INFO     Setting execution providers to                     
                                ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
                                in descending order of preference                  
   [10/08/25 00:43:33] INFO     Loading visual model 'ViT-B-32__openai' to memory  
   [10/08/25 00:43:33] INFO     Setting execution providers to                     
                                ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
                                in descending order of preference                  
   [10/08/25 00:43:33] INFO     Loading recognition model 'buffalo_l' to memory    
   [10/08/25 00:43:33] INFO     Setting execution providers to                     
                                ['CUDAExecutionProvider', 'CPUExecutionProvider'], 
                                in descending order of preference  
   

   If you see CUDAExecutionProvider and no errors, congratulations! You just set up your NVIDIA GPU with Immich for ML.

To verify that the GPU is being used, you can run nvidia-smi on the host and check the active processes while uploading images and videos to Immich:

> # nvidia-smi                                                                                                     
Tue Oct  7 02:23:51 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.163.01             Driver Version: 550.163.01     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060        On  |   00000000:2B:00.0 Off |                  N/A |
|  0%   50C    P8             14W /  170W |    1548MiB /  12288MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A   1577250      C   python                                       1542MiB |
+-----------------------------------------------------------------------------------------+

Troubleshooting

Here are a few quick checks and common fixes for ML/GPU setup issues.

No CUDAExecutionProvider in the execution provider list (/var/log/immich/ml.log)

Tip

Quick checks:

• Ensure onnxruntime-gpu (with CUDA and cuDNN extras) is installed inside the Immich ML venv.

source /opt/immich/app/machine-learning/ml-venv/bin/activate
uv pip list | grep 'onnx\|cuda\|cudnn'

• Confirm the ML logs show CUDAExecutionProvider after restarting the service:

tail -f --lines 100 /var/log/immich/ml.log

If CUDA isn’t listed, refer to the official onnxruntime-gpu documentation for required CUDA/cuDNN versions and setup:
https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#cuda-execution-provider

Failed to load library (/var/log/immich/ml.log)

Important

This usually indicates missing or incorrect library paths. Verify your environment variables in /etc/systemd/system/immich-ml.service under the [Service] section:

• PATH includes your CUDA bin directory (for example, /usr/local/cuda-12.x/bin).
• LD_LIBRARY_PATH includes the CUDA/cuDNN library directories from the ML venv.

After making changes, reload and restart the service:

systemctl daemon-reload
systemctl restart immich-ml

If the problem persists, check recent service logs for detailed errors:

journalctl -u immich-ml -n 200 --no-pager

Specify the number of threads explicitly so the affinity is not set. (/var/log/immich/ml.log)

Check this: #8193 (reply in thread)

• One other possible fix is to check
  nvidia-smi on the host and ensure that the GPU is not being currently used by any other LXC.

No space left on device

Increase storage of the LXC by running this command on the PVE host:

pve resize <CTID> rootfs +<storage to add>

For example, to increase LXC 100 storage by 8GB:

pve resize 100 rootfs +8G

Notes

It would be really neat to have Community Scripts ask and configure the
onnxruntime-gpu package with user-selected extras for GPU-supported ML features when initializing the LXC.

---

Revisions

• Edited for formatting, minor corrections
• (Oct 11): Added steps for installing cuda-toolkit, updated troubleshooting

[vhsdream]

Hey this is great - thanks for making this guide! I happen to have a NVIDIA GPU (but I find using my Intel iGPU is more efficient) so I'm going to test this ann report back any issues. As for adding the option to have this configured during installation, I'm all for having it added as long as it is shown to work without too much fuss.

[lenne0815]

Awesome guide ! I did all the steps you described and got until the point of actually running the ML, unfortunately
I now get "error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set" when immich is trying to start to use the gpu, do you have any idea where i can fix that ?

[adityataps]

@lenne0815 You're correct - we needed cuda-toolkit to get onnxruntime working. I updated the guide to add the step to install cuda-toolkit. Check out Step 6 in the ML section and let me know if these steps work for you!

[lenne0815]

Thank you for checking ! I had another problem, unrelated to your guide, the latest nvid driver apt pulls is 535 for me because im still on a 1080 so i had to clean the host and the lxc, I will try to use your driver and cuda version now ! ( The lowest driver version possible according to immich docs is: ""The installed driver must be >= 545 (it must support CUDA 12.3)"""

[adityataps]

I would also check your Debian version in the LXC and on the host. Debian 13 I believe has 550 as the latest supported version.
Looking at the NVIDIA drivers page it looks like 1080 should easily support version 550 since the latest supported driver is at 580, unless I'm mistaken

[lenne0815]

I tried three more times yesterday, with installing cuda toolkit the lxc already hits more than 20gb so im guessing my actual problem is somewhere there (env paths dont work even though they should be ok) Im trying again right now.

[lenne0815]

So today I tried again and got stuck at the same stage, i think i got driver / cuda etc installed cortrectly but for some strange reason the immich service recons it cant start because it cant read my LD_LIBRARY_PATH. I double and triple checked through the paths do seem to be correct.

The output of { echo "/usr/local/cuda-12.4/lib64"; find /opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia -type d -name "lib"; } | paste -sd:

is this one:

Environment="LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cufft/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cublas/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cuda_runtime/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cudnn/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/curand/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/nvjitlink/lib:/opt/immich/app/machine-learning/ml-venv/lib/python3.11/site-packages/nvidia/cuda_nvrtc/lib"

The error:

root@immich:~# tail -f --lines 100 /var/log/immich/ml.log
[10/13/25 15:19:49] INFO Starting gunicorn 23.0.0
[10/13/25 15:19:49] INFO Listening at: http://[::]:3003 (86570)
[10/13/25 15:19:49] INFO Using worker: immich_ml.config.CustomUvicornWorker
[10/13/25 15:19:49] INFO Booting worker with pid: 86574
[10/13/25 15:19:51] INFO Started server process [86574]
[10/13/25 15:19:51] INFO Waiting for application startup.
[10/13/25 15:19:51] INFO Created in-memory cache with unloading after 300s
of inactivity.
[10/13/25 15:19:51] INFO Initialized request thread pool with 4 threads.
[10/13/25 15:19:51] INFO Application startup complete.
[10/13/25 16:55:55] INFO Shutting down
[10/13/25 16:55:55] INFO Handling signal: term
[10/13/25 16:55:56] INFO Waiting for application shutdown.
[10/13/25 16:55:57] INFO Application shutdown complete.
[10/13/25 16:55:57] INFO Finished server process [86574]
[10/13/25 16:55:57] ERROR Worker (pid:86574) was sent SIGTERM!
[10/13/25 16:55:57] INFO Shutting down: Master
[10/13/25 16:56:07] INFO Starting gunicorn 23.0.0
[10/13/25 16:56:07] INFO Listening at: http://[::]:3003 (335)
[10/13/25 16:56:07] INFO Using worker: immich_ml.config.CustomUvicornWorker
[10/13/25 16:56:07] INFO Booting worker with pid: 337
[10/13/25 16:56:11] INFO Started server process [337]
[10/13/25 16:56:11] INFO Waiting for application startup.
[10/13/25 16:56:11] INFO Created in-memory cache with unloading after 300s
of inactivity.
[10/13/25 16:56:11] INFO Initialized request thread pool with 4 threads.
[10/13/25 16:56:11] INFO Application startup complete.
[10/13/25 17:18:24] INFO Shutting down
[10/13/25 17:18:24] INFO Handling signal: term
[10/13/25 17:18:25] INFO Waiting for application shutdown.
[10/13/25 17:18:28] INFO Application shutdown complete.
[10/13/25 17:18:28] INFO Finished server process [337]
[10/13/25 17:18:28] ERROR Worker (pid:337) was sent SIGTERM!
[10/13/25 17:18:28] INFO Shutting down: Master
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory
env: ‘bash’: No such file or directory

And:

root@immich:~# journalctl -u immich-ml -n 200 --no-pager
Oct 13 15:19:48 immich systemd[1]: Started immich-ml.service - immich Machine-Learning.
Oct 13 16:55:55 immich systemd[1]: Stopping immich-ml.service - immich Machine-Learning...
Oct 13 16:55:57 immich systemd[1]: immich-ml.service: Deactivated successfully.
Oct 13 16:55:57 immich systemd[1]: Stopped immich-ml.service - immich Machine-Learning.
Oct 13 16:55:57 immich systemd[1]: immich-ml.service: Consumed 11.403s CPU time, 198.2M memory peak, 89.2M memory swap peak.
-- Boot 6013c48e82ae494a82658224f804ce21 --
Oct 13 16:56:06 immich systemd[1]: Started immich-ml.service - immich Machine-Learning.
Oct 13 17:18:24 immich systemd[1]: Stopping immich-ml.service - immich Machine-Learning...
Oct 13 17:18:28 immich systemd[1]: immich-ml.service: Deactivated successfully.
Oct 13 17:18:28 immich systemd[1]: Stopped immich-ml.service - immich Machine-Learning.
Oct 13 17:18:28 immich systemd[1]: immich-ml.service: Consumed 6.434s CPU time, 372.7M memory peak, 180.4M memory swap peak.
-- Boot cf6c47f0555646cc99bf9b6888eeb750 --
Oct 13 17:18:35 immich systemd[1]: Started immich-ml.service.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Main process exited, code=exited, status=127/n/a
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Failed with result 'exit-code'.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Scheduled restart job, restart counter is at 1.
Oct 13 17:18:35 immich systemd[1]: Started immich-ml.service.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Main process exited, code=exited, status=127/n/a
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Failed with result 'exit-code'.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Scheduled restart job, restart counter is at 2.
Oct 13 17:18:35 immich systemd[1]: Started immich-ml.service.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Main process exited, code=exited, status=127/n/a
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Failed with result 'exit-code'.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Scheduled restart job, restart counter is at 3.
Oct 13 17:18:35 immich systemd[1]: Started immich-ml.service.
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Main process exited, code=exited, status=127/n/a
Oct 13 17:18:35 immich systemd[1]: immich-ml.service: Failed with result 'exit-code'.
Oct 13 17:18:36 immich systemd[1]: immich-ml.service: Scheduled restart job, restart counter is at 4.
Oct 13 17:18:36 immich systemd[1]: Started immich-ml.service.
Oct 13 17:18:36 immich systemd[1]: immich-ml.service: Main process exited, code=exited, status=127/n/a
Oct 13 17:18:36 immich systemd[1]: immich-ml.service: Failed with result 'exit-code'.
Oct 13 17:18:36 immich systemd[1]: immich-ml.service: Scheduled restart job, restart counter is at 5.
Oct 13 17:18:36 immich systemd[1]: immich-ml.service: Start request repeated too quickly.
Oct 13 17:18:36 immich systemd[1]: immich-ml.service: Failed with result 'exit-code'.
Oct 13 17:18:36 immich systemd[1]: Failed to start immich-ml.service.