Docker-in-Docker fails: llama-swap can't access Docker socket after UID/GID update

[alhadebe]

🐛 Describe the bug

After the recent [commit f91a8b2](f91a8b2), llama-swap can no longer launch model containers when running Docker-in-Docker (mounting /var/run/docker.sock).

Error:

docker: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock:
Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.51/images/create?fromImage=ghcr.io%2Fggml-org%2Fllama.cpp&tag=server-cuda":
dial unix /var/run/docker.sock: connect: permission denied

Everything worked correctly before this commit introduced the non-root (UID 10001) runtime user.

---

✅ Expected behaviour

llama-swap should still be able to start model containers via the mounted Docker socket when running inside Docker (DinD mode), just as it did prior to the UID/GID change.

---

🧰 Operating system and version

• OS: Ubuntu 24.04 LTS
• GPU: NVIDIA RTX 3090
• Docker: 27.1.1
• NVIDIA Container Toolkit: 1.16.2
• llama-swap image: ghcr.io/mostlygeek/llama-swap:cuda (latest post-f91a8b2)

---

⚙️ My Configuration

docker-compose.yml

services:
  llama-swap:
    container_name: llama-swap
    image: ghcr.io/mostlygeek/llama-swap:cuda
    network_mode: host
    restart: always
    volumes:
      - ./dockerdata/models:/models
      - ./dockerdata/llama-swap/config:/config
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
    pull_policy: always
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]

config.yaml (excerpt)

healthCheckTimeout: 300
logLevel: debug
startPort: 65001

models:
  "Qwen3-30B-A3B-Instruct-2507":
    cmd: |
      docker run --gpus=all --pull always --name Qwen3-30B-A3B-Instruct-2507
      --init --rm -p ${PORT}:8080 -v /home/al/dockerdata/models:/models ghcr.io/ggml-org/llama.cpp:server-cuda
      --model /models/Qwen_Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf
      --ctx-size 70000 --gpu-layers 999 --jinja
      --cache-type-k q8_0 --cache-type-v q8_0
      --temp 0.7 --top-p 0.8
    ttl: 300
    cmdStop: docker stop Qwen3-30B-A3B-Instruct-2507

---

📜 Proxy Logs

[error] dial unix /var/run/docker.sock: connect: permission denied
[debug] failed to start model container "Qwen3-30B-A3B-Instruct-2507"

📜 Upstream Logs

docker: permission denied while trying to connect to the Docker daemon socket...

---

🧾 Diagnostics

Inside the container:

# docker exec -it llama-swap ls -l /var/run/docker.sock
srw-rw---- 1 root 988 0 Nov  2 00:37 /var/run/docker.sock

Host docker group ID = 988
The internal non-root user (UID 10001) is not a member of that group, so it cannot open the socket.

---

[mostlygeek]

Hi, add uid 10001 to the docker group (998) that has read/write access to docker.sock.

[mdziekon]

After upgrading llama-swap (to v172-cuda-b6951) I've run into the same issue it seems.

Adding a new user to the host (sudo adduser --no-create-home --uid 10001 llama-swap-user) and adding it to the docker group (sudo usermod -a -G docker llama-swap-user) did not resolve the issue. Any help would be appreciated.

[mostlygeek]

try using: user:root in your docker compose.

[alhadebe]

user: root didn't seem to work for me.

[ryan-steed-usa]

That seems very strange, the root user should have full access. What happens if you run this?

docker exec -u root -it llama-swap bash -c 'ls -l /var/run/docker.sock; docker ps'

Edit: forgot -u root

[alhadebe]

This is the output

srw-rw---- 1 root 988 0 Nov 2 00:37 /var/run/docker.sock
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d679b2bb8a6b ghcr.io/mostlygeek/llama-swap:cuda "/app/llama-swap -co…" 20 hours ago Up About an hour (unhealthy) llama-swap

[alhadebe]

Scratch that user: root seems to be working now.

[ryan-steed-usa]

Excellent! If you'd like to attempt running the parent container as the app user, try adding the following to your docker compose:

    user: app:988
    group_add: app

This should run the container with a primary group GID of 988 (host docker group) while retaining app group supplementary membership: uid=10001(app) gid=988(988) groups=988(988),10001(app)

Edit: actually, it might be best to try adding the docker group as a supplementary GID:

    group_add: 988