From b7cf23177c498c6e8a610b75c80222f560b56081 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 07:01:10 -0600 Subject: [PATCH 1/3] fix(gpu): declare CUDA_RUNTIME_IMAGE before the first FROM MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GPU image builds have failed on every 5.2 beta since beta.2 with: ERROR: failed to build: failed to solve: base name (${CUDA_RUNTIME_IMAGE}) should not be blank An ARG referenced by a FROM must be declared in the global scope, before any stage. #496 added `ARG CUDA_RUNTIME_IMAGE` immediately above the `FROM ${CUDA_RUNTIME_IMAGE} AS run` line, but that position is after `FROM ... AS build`, so the ARG belonged to the build stage and the later FROM expanded it to an empty base name. Moves the ARG (with its comment) above the first FROM, alongside NODE_BUILD_VERSION and NODE_VERSION, which are already global for exactly this reason. The default and the override contract are unchanged — a T4 host pinned to an older driver branch still overrides with `--build-arg CUDA_RUNTIME_IMAGE=nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04`. Added a note on the ordering constraint so it does not regress. Only the GPU images were affected; standard, pointer-compression and OpenShift builds do not use this ARG and have been publishing normally. 5.1.x is unaffected — it has no CUDA_RUNTIME_IMAGE ARG at all. --- Dockerfile-gpu | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/Dockerfile-gpu b/Dockerfile-gpu index b5f9cd6ca..eddfbba29 100644 --- a/Dockerfile-gpu +++ b/Dockerfile-gpu @@ -1,6 +1,17 @@ ARG NODE_BUILD_VERSION=24 ARG NODE_VERSION=24 +# Runtime base. CUDA 13.x supports Turing (SM 7.5 / T4) — Turing is the minimum +# architecture in the 13.x line (Maxwell/Pascal/Volta were dropped). The 13.x +# runtime requires an R580+ host driver; on hosts pinned to an older driver +# branch, override with a CUDA 12.x cudnn-runtime base, e.g.: +# --build-arg CUDA_RUNTIME_IMAGE=nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04 +# +# Must stay above the first FROM: an ARG referenced by a FROM has to be declared +# in the global scope, before any stage. Declared after a FROM it belongs to that +# stage, and the later FROM expands it to an empty base name. +ARG CUDA_RUNTIME_IMAGE=nvidia/cuda:13.3.0-cudnn-runtime-ubuntu22.04 + FROM docker.io/node:${NODE_BUILD_VERSION} AS build WORKDIR /usr/src/harper-pro @@ -9,12 +20,6 @@ COPY . . RUN env NO_USE_GIT=true npm run package -# Runtime base. CUDA 13.x supports Turing (SM 7.5 / T4) — Turing is the minimum -# architecture in the 13.x line (Maxwell/Pascal/Volta were dropped). The 13.x -# runtime requires an R580+ host driver; on hosts pinned to an older driver -# branch, override with a CUDA 12.x cudnn-runtime base, e.g.: -# --build-arg CUDA_RUNTIME_IMAGE=nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04 -ARG CUDA_RUNTIME_IMAGE=nvidia/cuda:13.3.0-cudnn-runtime-ubuntu22.04 FROM ${CUDA_RUNTIME_IMAGE} AS run ARG NODE_VERSION=24 From 4373340ea9e036adda51741acb05c97b5bb06750 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 07:35:26 -0600 Subject: [PATCH 2/3] fix(gpu): use the ubuntu24.04 CUDA base so bundled uWS can load MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CUDA runtime base was ubuntu22.04 (glibc 2.35). This image bundles uWebSockets.js, copied from the build stage, and its prebuilt Linux binaries link against GLIBC_2.38 — so the addon could not load on that base. It is the same failure documented at the top of ./Dockerfile, which is why the standard image runs on Debian trixie rather than bookworm (glibc 2.36). Switches the default to nvidia/cuda:13.3.0-cudnn-runtime-ubuntu24.04 (glibc 2.39) and updates the CUDA 12.x override example to the matching ubuntu24.04 tag. All four tag combinations were confirmed present on Docker Hub. Documents the glibc floor next to the ARG: the point of this ARG is that operators override it for older driver branches, and an override back to an ubuntu22.04 base would silently break uWS again. Latent until now — the build has failed outright since #496, so this base was never actually produced. Credit: flagged by gemini-code-assist on #628. --- Dockerfile-gpu | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/Dockerfile-gpu b/Dockerfile-gpu index eddfbba29..7efaa6832 100644 --- a/Dockerfile-gpu +++ b/Dockerfile-gpu @@ -5,12 +5,18 @@ ARG NODE_VERSION=24 # architecture in the 13.x line (Maxwell/Pascal/Volta were dropped). The 13.x # runtime requires an R580+ host driver; on hosts pinned to an older driver # branch, override with a CUDA 12.x cudnn-runtime base, e.g.: -# --build-arg CUDA_RUNTIME_IMAGE=nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04 +# --build-arg CUDA_RUNTIME_IMAGE=nvidia/cuda:12.6.3-cudnn-runtime-ubuntu24.04 +# +# Keep the ubuntu24.04 variant (glibc 2.39) whichever CUDA line you pick. This +# image bundles uWebSockets.js (copied from the build stage below), whose +# prebuilt Linux binaries link against GLIBC_2.38 — ubuntu22.04 ships glibc 2.35 +# and the addon fails to load there, the same failure that put the standard +# image on Debian trixie. See the note at the top of ./Dockerfile. # # Must stay above the first FROM: an ARG referenced by a FROM has to be declared # in the global scope, before any stage. Declared after a FROM it belongs to that # stage, and the later FROM expands it to an empty base name. -ARG CUDA_RUNTIME_IMAGE=nvidia/cuda:13.3.0-cudnn-runtime-ubuntu22.04 +ARG CUDA_RUNTIME_IMAGE=nvidia/cuda:13.3.0-cudnn-runtime-ubuntu24.04 FROM docker.io/node:${NODE_BUILD_VERSION} AS build From 9c4b47e059866ad7799132d38250e71f6d6585de Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Wed, 29 Jul 2026 07:39:57 -0600 Subject: [PATCH 3/3] fix(gpu): reclaim uid 1000 for harperdb, and fail the setup block loudly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two problems the ubuntu24.04 base exposed. 1. Ubuntu 24.04 images ship a default `ubuntu` user at uid/gid 1000, so `groupadd --gid 1000` / `useradd --uid 1000` failed: groupadd: GID 1000 already exists useradd: UID 1000 is not unique harperdb has to own 1000 — existing deployments chown their data volumes to it — so rename the incumbent rather than moving harperdb to another id. This is what ./Dockerfile already does for the node image's `node` user. Handles both shapes, since CUDA_RUNTIME_IMAGE is overridable and 22.04 bases ship no uid-1000 user. 2. The setup block had no `set -e`, so both failures above were swallowed: the block continued and exited 0 on the trailing `rm -rf`, producing an image with no harperdb user. That surfaced much later and far from the cause, as unable to find user harperdb: no matching entries in passwd file on the pnpm step. Adding `set -e` makes the block fail at the real cause. Verified the reclaim logic under sh with both base shapes (uid 1000 present and absent) — the rename path and the create path each exit 0. --- Dockerfile-gpu | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/Dockerfile-gpu b/Dockerfile-gpu index 7efaa6832..34efdae55 100644 --- a/Dockerfile-gpu +++ b/Dockerfile-gpu @@ -49,8 +49,23 @@ RUN <<-EOF | tee /etc/apt/sources.list.d/nodesource.list apt-get update apt-get install -y nodejs - groupadd --gid 1000 harperdb - useradd --uid 1000 --gid 1000 --home-dir /home/harperdb --shell /bin/bash --no-create-home harperdb + # harperdb must own uid/gid 1000 — existing deployments' data volumes are + # chowned to it — so reclaim the id from any incumbent rather than moving + # harperdb. Ubuntu 24.04 bases ship a default `ubuntu` user at 1000; 22.04 + # bases ship none, and CUDA_RUNTIME_IMAGE is overridable, so handle both. + # Renaming (as ./Dockerfile does for node:*'s `node` user) keeps the id stable. + EXISTING_USER="$(getent passwd 1000 | cut -d: -f1)" || true + if [ -n "$EXISTING_USER" ]; then + EXISTING_GROUP="$(getent group 1000 | cut -d: -f1)" || true + usermod -d /home/harperdb -l harperdb "$EXISTING_USER" + if [ -n "$EXISTING_GROUP" ]; then + groupmod -n harperdb "$EXISTING_GROUP" + fi + rm -rf "/home/$EXISTING_USER" + else + groupadd --gid 1000 harperdb + useradd --uid 1000 --gid 1000 --home-dir /home/harperdb --shell /bin/bash --no-create-home harperdb + fi mkdir -p /home/harperdb chown harperdb:harperdb /home/harperdb rm -rf /var/lib/apt/lists/*