perf(docker): dispatch CPU kernels at runtime and unify the three images

.dockerignore

@@ -1,27 +1,19 @@
-.venv
-.pytest_cache
-.ruff_cache
-.mypy_cache
-__pycache__
-tests
-data
-*.db
-*.sqlite*
-*.wav
-*.log
+# Allowlist, not a denylist. The build copies exactly four things — pyproject.toml,
+# uv.lock, README.md (hatchling reads it for the wheel metadata) and app/ — so
+# everything else is excluded up front and re-admitted below. A denylist has to be
+# amended every time the repo grows a directory; this does not, and it keeps the
+# context small enough that a build never waits on it.
+#
+# A populated .env holds the bearer token. No stage copies it, but a `*` rule means
+# it is never sent to the daemon or captured in a build cache layer either.
+*
 
-# A populated .env holds the bearer token. No Dockerfile copies it, but keeping
-# it out of the build context means it is never sent to the daemon or captured
-# in a build cache layer.
-.env
-.envrc
+!pyproject.toml
+!uv.lock
+!README.md
+!app
+!app/**
 
-# Never needed by a build stage; .git in particular is large and changes on
-# every commit, which would invalidate the context for no reason.
-.git
-.gitignore
-.github
-docs
-scripts
-LICENSE
-justfile
+# Re-exclude what the allowlist above lets back in through app/.
+app/**/__pycache__
+app/**/*.pyc

.env.example

@@ -27,7 +27,7 @@
 #   1. Required: bearer token
 #   2. Where Docker publishes the gateway
 #   3. Address the pairing QR gives the phone
-#   4. Which image to run
+#   4. Which image to run, and how it is built
 #   5. Gateway behaviour
 #   6. Listener inside the container
 #   7. Settings that do not belong here
@@ -103,7 +103,7 @@ VOCAGATEWAY_PUBLISH_PORT=8765
 
 
 # ============================================================================
-# 4. Which image to run
+# 4. Which image to run, and how it is built
 # ============================================================================
 
 # Tag for the `gateway` service. Setting it does not by itself switch Compose
@@ -114,9 +114,51 @@ VOCAGATEWAY_PUBLISH_PORT=8765
 #   docker compose pull
 #   docker compose up --detach --no-build
 #
-# The native/cuda/vulkan profile services carry fixed tags and ignore this.
+# The cuda/vulkan profile services carry fixed tags and ignore this.
 #VOCAGATEWAY_IMAGE=ghcr.io/your-user/vocagateway:latest
 
+# Extra cmake flags for the whisper.cpp build, appended after the Dockerfile's
+# own, so a flag here overrides the default. Unlike the git-commit build args in
+# section 7, this one belongs in .env: it is a property of the machine you build
+# for, not of the moment you built.
+#
+# The two that come up:
+#
+#   -DCMAKE_CUDA_ARCHITECTURES=89-real
+#       Build CUDA kernels for one known GPU (89 = Ada / RTX 40-series) instead
+#       of the portable spread ggml picks. Much faster nvcc and a far smaller
+#       binary; the image then runs only on that GPU generation.
+#
+#   -DGGML_BLAS=OFF
+#       Drop OpenBLAS from the CPU image. Measured on arm64 this is a ~1.6x
+#       slowdown (tiny.en on an 11 s clip: ~2.0 s with BLAS, ~3.2 s without), so
+#       BLAS is on by default. If your host is x86 with AVX-512 the balance may
+#       differ; there is a timing recipe under "Tuning the whisper.cpp build" in
+#       docs/deployment.md.
+#
+# There is no CPU "native" profile to set here. The image builds every ggml CPU
+# micro-architecture variant and loads the best one the host reports at startup.
+#VOCAGATEWAY_WHISPER_CMAKE_EXTRA=-DCMAKE_CUDA_ARCHITECTURES=89-real
+
+# How many compile jobs the whisper.cpp build runs at once. Blank is resolved to
+# the builder's CPU count. The cuda image is the usual reason to set a lower
+# value: nvcc instantiates a great many templates and each job can want most of a
+# gigabyte, so a memory-constrained build can be OOM-killed partway through.
+# Roughly one job per 2 GB of builder memory. An 8 GB machine normally wants 3.
+#VOCAGATEWAY_BUILD_JOBS=3
+
+# Supplementary groups the vulkan profile's container joins so uid 10001 can
+# open /dev/dri/renderD128. Without the render group the open fails with EACCES,
+# Vulkan finds no device, and whisper.cpp silently falls back to Mesa's software
+# rasteriser — slower than the CPU image. The GID differs per distribution (993
+# on Ubuntu, 104 on Debian); read your host's with:
+#
+#   stat -c '%g' /dev/dri/renderD128
+#
+# Ignored by the default and cuda services.
+#VOCAGATEWAY_RENDER_GID=993
+#VOCAGATEWAY_VIDEO_GID=44
+
 
 # ============================================================================
 # 5. Gateway behaviour

.github/workflows/container.yml

@@ -3,26 +3,30 @@ name: Container
 # Building the image compiles whisper.cpp from source, which is by far the most
 # expensive thing in CI. Nothing under app/ can break that build — the runtime
 # image only copies the package, it never imports it — so this runs on the files
-# that actually decide whether the image assembles: the Dockerfiles and the
-# dependency set they install.
+# that actually decide whether the image assembles: the Dockerfile and the
+# dependency set it installs.
+#
+# One Dockerfile now builds all three accelerator images, so a change to it can
+# break cuda or vulkan while cpu still succeeds. The matrix below builds each
+# one. They are the reason this belongs in CI rather than on a laptop. A
+# portable cuda image compiles kernels for a broad GPU architecture spread; the
+# matrix narrows its compile-only validation to one architecture because no GPU
+# executes the cache-only result. Release/operator builds keep the portable
+# Dockerfile default.
 on:
   push:
     branches: [main]
-    paths:
-      - 'Dockerfile*'
+    paths: &container-paths
+      - 'Dockerfile'
       - '.dockerignore'
+      - 'compose.yaml'
       - 'pyproject.toml'
       - 'uv.lock'
       - '.github/workflows/container.yml'
   pull_request:
     branches: [main]
     types: [opened, synchronize, reopened, ready_for_review]
-    paths:
-      - 'Dockerfile*'
-      - '.dockerignore'
-      - 'pyproject.toml'
-      - 'uv.lock'
-      - '.github/workflows/container.yml'
+    paths: *container-paths
   workflow_dispatch:
 
 permissions:
@@ -33,28 +37,171 @@ concurrency:
   cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
-  container:
+  build:
     if: github.event_name != 'pull_request' || !github.event.pull_request.draft
     runs-on: ubuntu-24.04
-    timeout-minutes: 30
+    timeout-minutes: ${{ matrix.timeout }}
+    strategy:
+      # One accelerator failing says nothing about the others, and the cuda job
+      # is long enough that cancelling it on a fast vulkan failure wastes the
+      # whole run.
+      fail-fast: false
+      matrix:
+        include:
+          # The cpu image is the one every deployment gets by default, so it is
+          # also the one that is loaded and actually run below.
+          - accel: cpu
+            timeout: 40
+            smoke: true
+            reclaim-disk: false
+            build_jobs: 4
+            cmake_extra: ''
+          - accel: vulkan
+            timeout: 40
+            smoke: false
+            reclaim-disk: false
+            build_jobs: 4
+            cmake_extra: ''
+          # The CUDA devel/runtime bases and Python engine layers are still much
+          # larger than the other variants, so reclaim the runner's unused SDKs.
+          - accel: cuda
+            timeout: 40
+            smoke: false
+            reclaim-disk: true
+            build_jobs: 3
+            # Compiling every CUDA architecture dominates the job despite the
+            # image never running on a GPU. Ada is a representative real-code
+            # compile; CPU dispatch is already built and exercised above.
+            cmake_extra: >-
+              -DCMAKE_CUDA_ARCHITECTURES=89-real
+              -DGGML_CPU_ALL_VARIANTS=OFF
+    name: build (${{ matrix.accel }})
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
           persist-credentials: false
+
+      - name: Reclaim runner disk space
+        if: matrix.reclaim-disk
+        run: |
+          # Preinstalled SDKs this build never touches. Worth ~25 GB, which is
+          # the difference between the cuda image building and the runner
+          # filling up partway through.
+          sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc
+          sudo rm -rf /usr/local/share/boost "${AGENT_TOOLSDIRECTORY}"
+          df -h /
+
       - uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
-      - name: Build gateway image
+
+      - name: Build ${{ matrix.accel }} image
         run: |
-          # cacheonly skips exporting the assembled image to the local daemon.
-          # The question this job answers is whether the build succeeds, and
-          # nothing downstream runs the image.
+          # A smoke-tested image has to reach the local daemon; the others only
+          # need to prove they assemble, so they stop at the cache.
+          if [ "${SMOKE}" = "true" ]; then
+            output="type=docker"
+          else
+            output="type=cacheonly"
+          fi
           # Only the sha is stamped here: a commit subject is attacker-supplied
           # text on a pull request, and expanding it into this shell would be an
           # injection. Release builds that publish an image add the subject from
           # a checkout instead.
           docker buildx build \
-            --tag vocagateway:test \
+            --tag "vocagateway:test-${ACCEL}" \
+            --build-arg "ACCEL=${ACCEL}" \
+            --build-arg "BUILD_JOBS=${BUILD_JOBS}" \
+            --build-arg "WHISPER_CMAKE_EXTRA=${CMAKE_EXTRA}" \
             --build-arg VOCAGATEWAY_GIT_COMMIT="${GITHUB_SHA}" \
-            --cache-from type=gha,scope=gateway \
-            --cache-to type=gha,mode=max,scope=gateway \
-            --output type=cacheonly \
+            --cache-from "type=gha,scope=gateway-${ACCEL}" \
+            --cache-to "type=gha,mode=max,scope=gateway-${ACCEL}" \
+            --output "${output}" \
             .
+        env:
+          ACCEL: ${{ matrix.accel }}
+          BUILD_JOBS: ${{ matrix.build_jobs }}
+          CMAKE_EXTRA: ${{ matrix.cmake_extra }}
+          SMOKE: ${{ matrix.smoke }}
+
+      # Everything below is the cpu image only. It is what a laptop build was
+      # being used to check by hand, and it catches the failures a successful
+      # `docker build` does not: a backend that is present but never loaded, a
+      # wheel that is missing its templates, a container that cannot start under
+      # the hardening compose applies.
+      - name: Check ggml loads a CPU backend variant
+        if: matrix.smoke
+        run: |
+          set -euo pipefail
+          curl --fail --location --silent --show-error \
+            --output jfk.wav \
+            https://raw.githubusercontent.com/ggml-org/whisper.cpp/v1.9.1/samples/jfk.wav
+          curl --fail --location --silent --show-error \
+            --output ggml-tiny.en.bin \
+            https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny.en.bin
+
+          docker run --rm --volume "${PWD}:/bench:ro" \
+            --entrypoint whisper-cli vocagateway:test-cpu \
+            -m /bench/ggml-tiny.en.bin -f /bench/jfk.wav -nt -t 2 -bs 2 -bo 2 \
+            > transcript.txt 2> backend.log || { cat backend.log; exit 1; }
+
+          grep 'load_backend: loaded CPU backend' backend.log
+
+          # GGML_CPU_ALL_VARIANTS exists so a portable image still runs the
+          # host's instruction set. A GitHub runner is well past Haswell, so
+          # loading the x64 baseline means dispatch silently regressed and every
+          # deployment is running scalar kernels.
+          if grep -q 'libggml-cpu-x64\.so' backend.log; then
+            echo "::error::ggml fell back to the baseline x64 CPU backend" >&2
+            cat backend.log >&2
+            exit 1
+          fi
+
+          grep -qi 'ask not what your country can do for you' transcript.txt
+
+      - name: Check the container serves under compose's hardening
+        if: matrix.smoke
+        run: |
+          set -euo pipefail
+          # The same restrictions compose.yaml applies, so a capability the
+          # gateway turns out to need fails here rather than on a deployment.
+          docker run --detach --name vg-smoke --publish 18765:8765 \
+            --cap-drop ALL \
+            --security-opt no-new-privileges:true \
+            --tmpfs /tmp:size=64m,mode=1777 \
+            vocagateway:test-cpu
+
+          for _ in $(seq 1 30); do
+            status="$(docker inspect --format '{{.State.Health.Status}}' vg-smoke)"
+            [ "${status}" = "healthy" ] && break
+            [ "${status}" = "unhealthy" ] && break
+            sleep 2
+          done
+
+          if [ "${status}" != "healthy" ]; then
+            echo "::error::container never became healthy (${status})" >&2
+            docker logs vg-smoke >&2
+            exit 1
+          fi
+
+          curl --fail --silent --show-error --output /dev/null http://127.0.0.1:18765/health/live
+          # The WebUI renders Jinja templates from the installed wheel, so a 200
+          # here is what proves the non-editable install carried app/templates.
+          curl --fail --silent --show-error --output /dev/null http://127.0.0.1:18765/
+
+      - name: Container logs
+        if: failure() && matrix.smoke
+        run: docker logs vg-smoke || true
+
+  compose:
+    if: github.event_name != 'pull_request' || !github.event.pull_request.draft
+    runs-on: ubuntu-24.04
+    timeout-minutes: 5
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - name: Validate every profile interpolates
+        run: |
+          # `just compose` checks the default service. The profile services carry
+          # the device wiring, so they need naming explicitly to be parsed.
+          VOCAGATEWAY_TOKEN=test-token-with-at-least-thirty-two-characters \
+            docker compose --profile cuda --profile vulkan config --quiet

AGENTS.md

@@ -119,9 +119,9 @@ Docker Desktop cannot use macOS MLX / WhisperKit / Core ML. Host-only engines
 (`vocamac`, `handy`, `mlx-audio`, `whisperkit`) are hidden in the WebUI on Linux
 and in containers; API select is `422 invalid_engine`. Compose default publish is
 `127.0.0.1:8765`. `VOCAGATEWAY_NETWORK_MODE=host` is Linux Docker Engine only
-(not Docker Desktop) and ignores `VOCAGATEWAY_PUBLISH_HOST`/`PORT`. Profiles
-`native` / `cuda` / `vulkan` share the same port and `vocagateway-data` volume —
-run one service at a time.
+(not Docker Desktop) and ignores `VOCAGATEWAY_PUBLISH_HOST`/`PORT`. The default
+CPU service and the `cuda` / `vulkan` profile services share the same port and
+`vocagateway-data` volume — run one service at a time.
 
 ## Consumers
 
@@ -160,7 +160,7 @@ behind a reverse proxy at a **domain root**, not a subpath.
 | Workflow | When | What |
 | --- | --- | --- |
 | `quality.yml` | `app/`, `tests/`, `scripts/`, `pyproject.toml`, `uv.lock`, `compose.yaml` | ffmpeg, ruff, format `--check`, `mypy app`, pytest, compose config |
-| `container.yml` | `Dockerfile*`, `.dockerignore`, `pyproject.toml`, `uv.lock` | `docker buildx` of `Dockerfile` only (`cacheonly`; not CUDA/Vulkan) |
+| `container.yml` | `Dockerfile`, `.dockerignore`, `compose.yaml`, `pyproject.toml`, `uv.lock`, workflow | `docker buildx` CPU/CUDA/Vulkan matrix (representative CUDA architecture), CPU backend/runtime smoke test, all-profile Compose config |
 | `verify-model-pins.yml` | pin/catalog/harvester paths + weekly | `scripts/verify-model-pins.py` |
 | `deploy-pages.yml` | `web/**` on `main` | GitHub Pages |
 
@@ -177,7 +177,7 @@ hostnames. Diagnostics omit those. Keep `.env` and token files local.
 - Never commit or push to `main`. Do not merge PRs; wait for review.
 - PR body should match [`.github/pull_request_template.md`](.github/pull_request_template.md):
   **Summary** (what/why), **Verification** (`just test` or the equivalent ruff /
-  mypy / pytest / compose checks; container build if Dockerfiles or the lockfile
+  mypy / pytest / compose checks; container build if the Dockerfile or lockfile
   changed; docs if setup/network/config changed), **Privacy and security**
   checklist.
 - One logical change per PR.