Skip to content

Commit 139dbbd

Browse files
authored
Merge pull request #990 from shivamerla/add_cloud_build
Add GCB (Google Cloud Build) configuration to enable image building and promotion via the k8s-staging-images infrastructure, following the same pattern as dra-driver-cpu
2 parents a7a1caa + 7cbe9b6 commit 139dbbd

12 files changed

Lines changed: 95 additions & 18 deletions

File tree

.github/workflows/chart.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ jobs:
4444
sed -i "s/${VERSION:1}/${VERSION_W_COMMIT:1}-chart/g" deployments/helm/nvidia-dra-driver-gpu/Chart.yaml
4545
sed -i "s/tag: \"\"/tag: \"${VERSION_W_COMMIT}\"/g" deployments/helm/nvidia-dra-driver-gpu/values.yaml
4646
# Staging image registry for CI-built images.
47-
sed -i 's|registry.k8s.io/nvidia|gcr.io/k8s-staging-nvidia|g' deployments/helm/nvidia-dra-driver-gpu/values.yaml
47+
sed -i 's|registry.k8s.io/nv-dra-driver-gpu|us-central1-docker.pkg.dev/k8s-staging-images/nv-dra-driver-gpu|g' deployments/helm/nvidia-dra-driver-gpu/values.yaml
4848
git diff || echo "ignore git diff exit code"
4949
helm package deployments/helm/nvidia-dra-driver-gpu
5050
- name: Upload chart package

.github/workflows/image.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ jobs:
3939
# For now, we want to test multi-arch builds in GitHub Actions as well, but without pushing to the registry.
4040
- name: Multi-arch build
4141
env:
42-
IMAGE_NAME: registry.k8s.io/nvidia/nvidia-dra-driver-gpu
42+
IMAGE_NAME: registry.k8s.io/nv-dra-driver-gpu/nvidia-dra-driver-gpu
4343
BUILD_MULTI_ARCH_IMAGES: "true"
4444
BUILD_MULTI_ARCH_GH: "true"
4545
CI: "true"

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -246,4 +246,4 @@ bats-gpu:
246246
.PHONY: image-build-and-copy-to-nodes
247247
image-build-and-copy-to-nodes:
248248
make -f deployments/container/Makefile build
249-
bash hack/copy-image-to-k8s-nodes.sh registry.k8s.io/nvidia/nvidia-dra-driver-gpu:$(VERSION)
249+
bash hack/copy-image-to-k8s-nodes.sh $(REGISTRY)/$(DRIVER_NAME):$(VERSION)

RELEASE.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,18 +36,18 @@ After the release tag is pushed, the images must be built, pushed to staging, an
3636
- The Prow job configuration is in [test-infra](https://github.com/kubernetes/test-infra/blob/master/config/jobs/image-pushing/k8s-staging-nvidia-dra-driver-gpu.yaml) if troubleshooting is needed.
3737
- Verify the images exist in the staging repository:
3838
```sh
39-
skopeo list-tags docker://us-central1-docker.pkg.dev/k8s-staging-images/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu | grep vX.Y.Z
39+
skopeo list-tags docker://us-central1-docker.pkg.dev/k8s-staging-images/nv-dra-driver-gpu/nvidia-dra-driver-gpu | grep vX.Y.Z
4040
```
4141

4242
### B. Create PR for Promotion
4343

4444
- Identify the image digests:
4545
```sh
46-
skopeo inspect docker://us-central1-docker.pkg.dev/k8s-staging-images/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu:vX.Y.Z --format '{{.Digest}}'
46+
skopeo inspect docker://us-central1-docker.pkg.dev/k8s-staging-images/nv-dra-driver-gpu/nvidia-dra-driver-gpu:vX.Y.Z --format '{{.Digest}}'
4747
```
4848
- Fork [kubernetes/k8s.io](https://github.com/kubernetes/k8s.io).
4949
- Update
50-
`registry.k8s.io/images/k8s-staging-nvidia-dra-driver-gpu/images.yaml`
50+
`registry.k8s.io/images/k8s-staging-nv-dra-driver-gpu/images.yaml`
5151
with the new digests and tags ([sample PR](https://github.com/kubernetes/k8s.io/pull/9176)).
5252
- Submit the PR to `kubernetes/k8s.io`. Once it is approved and merged
5353
automation will schedule the promotion.
@@ -60,7 +60,7 @@ Before publishing the release, verify the images are available at k8s-registry:
6060

6161
- Ensure the images are available at `registry.k8s.io`:
6262
```sh
63-
skopeo list-tags docker://registry.k8s.io/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu | grep vX.Y.Z
63+
skopeo list-tags docker://registry.k8s.io/nv-dra-driver-gpu/nvidia-dra-driver-gpu | grep vX.Y.Z
6464
```
6565

6666
### B. Test Release Artifacts
@@ -73,7 +73,7 @@ Before publishing the release, verify the images are available at k8s-registry:
7373
- Generate the release manifests:
7474
```sh
7575
rm -rf ./dist
76-
REGISTRY=registry.k8s.io TAG=vX.Y.Z make manifests
76+
REGISTRY=registry.k8s.io/nv-dra-driver-gpu TAG=vX.Y.Z make manifests
7777
ls ./dist # to confirm the generated artifacts
7878
```
7979
- Run a local E2E test in Kind (see references in `README.md`) using the generated manifests from the `dist/` directory to ensure they pull the correct images and function as expected.

cloudbuild.yaml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# See https://cloud.google.com/cloud-build/docs/build-config
2+
#
3+
# Invoked by Kubernetes Prow (image-builder) on merges/tags; see
4+
# config/jobs/image-pushing/k8s-staging-nvidia-dra-driver-gpu.yaml in kubernetes/test-infra.
5+
# Image tags follow versions.mk VERSION_W_COMMIT (not image-builder _GIT_TAG).
6+
7+
timeout: 7200s
8+
options:
9+
substitutionOption: ALLOW_LOOSE
10+
steps:
11+
- name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20260127-c1affcc8de
12+
entrypoint: ./scripts/build-and-publish-image.sh
13+
env:
14+
- IMG_PREFIX=us-central1-docker.pkg.dev/k8s-staging-images/nv-dra-driver-gpu
15+
- GIT_COMMIT=${_PULL_BASE_SHA}
16+
substitutions:
17+
_GIT_TAG: 'v0-placeholder'
18+
_PULL_BASE_REF: 'main'
19+
_PULL_BASE_SHA: '1234567890abcdef0123456789abcdef01234567'

deployments/container/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ RUN bash /bashbuild/build-bash-static.sh
104104
RUN cd /bashbuild/bash-static/releases && ./bash*-static --version && mv bash-*-static /bashbuild/bash && file /bashbuild/bash
105105

106106
# option 2:
107-
# FROM registry.k8s.io/nvidia/nvidia-dra-driver-gpu:v25.12.0-dev-839e966a AS bash
107+
# FROM registry.k8s.io/nv-dra-driver-gpu/nvidia-dra-driver-gpu:v25.12.0-dev-839e966a AS bash
108108

109109
# Pull the nvidia-cdi-hook binary out of the relevant toolkit container
110110
# (arch: TARGETPLATFORM, set via --platform).

deployments/helm/nvidia-dra-driver-gpu/values.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ resourceApiVersion: ""
5050

5151
imagePullSecrets: []
5252
image:
53-
repository: registry.k8s.io/nvidia/nvidia-dra-driver-gpu
53+
repository: registry.k8s.io/nv-dra-driver-gpu/nvidia-dra-driver-gpu
5454
pullPolicy: IfNotPresent
5555
# Note: an empty string is translated to the `appVersion` string from
5656
# the Helm chart YAML (effectively implementing the default value to be

hack/copy-image-to-k8s-nodes.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ set -o nounset
44
set -o pipefail
55

66
if [ $# -ne 1 ]; then
7-
echo "first arg: required: image ref, example: registry.k8s.io/nvidia/nvidia-dra-driver-gpu:v25.12.0-dev"
7+
echo "first arg: required: image ref, example: registry.k8s.io/nv-dra-driver-gpu/nvidia-dra-driver-gpu:v25.12.0-dev"
88
exit 1
99
else
1010
export CONT_IMAGE_REF="$1"

scripts/build-and-publish-image.sh

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
#!/usr/bin/env bash
2+
# Copyright The Kubernetes Authors.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
set -euo pipefail
16+
17+
REPO_ROOT=$(git rev-parse --show-toplevel)
18+
cd "$REPO_ROOT"
19+
20+
if [[ -z ${IMG_PREFIX:-} ]]; then
21+
echo "IMG_PREFIX is not set"
22+
exit 1
23+
fi
24+
25+
if [[ -z ${IMG_TAG:-} ]]; then
26+
# Match versions.mk VERSION_W_COMMIT (e.g. v25.8.0-dev-f2eaddd6) for release
27+
# commits, use the exact git tag as the image tag.
28+
if git describe --exact-match --tags HEAD >/dev/null 2>&1; then
29+
IMG_TAG=$(git describe --exact-match --tags HEAD)
30+
else
31+
IMG_TAG=$(make --no-print-directory -f "${REPO_ROOT}/versions.mk" print-VERSION_W_COMMIT)
32+
fi
33+
fi
34+
echo "Using IMG_TAG=${IMG_TAG}"
35+
36+
if [[ -z ${GIT_COMMIT:-} ]]; then
37+
GIT_COMMIT=$(git rev-parse HEAD)
38+
fi
39+
echo "Using GIT_COMMIT=${GIT_COMMIT}"
40+
41+
export CI=true
42+
export DOCKER_CLI_EXPERIMENTAL=enabled
43+
44+
# Register gcloud as a Docker credential helper.
45+
# Required for "docker buildx build --push".
46+
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
47+
48+
docker buildx create --name nv-dra-driver-gpu-builder || true
49+
docker buildx use nv-dra-driver-gpu-builder
50+
docker buildx inspect --bootstrap
51+
52+
make -f deployments/container/Makefile build \
53+
BUILD_MULTI_ARCH_IMAGES=true \
54+
PUSH_ON_BUILD=true \
55+
REGISTRY="${IMG_PREFIX}" \
56+
VERSION="${IMG_TAG}" \
57+
GIT_COMMIT="${GIT_COMMIT}"

tests/bats/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ TEST_NVIDIA_DRIVER_ROOT ?= "/run/nvidia/driver"
4343

4444
# TODO: calculate precise, expected container image spec,
4545
# and test for that in test suite, for example:
46-
# `registry.k8s.io/nvidia/nvidia-dra-driver-gpu:v25.8.0-dev`
46+
# `registry.k8s.io/nv-dra-driver-gpu/nvidia-dra-driver-gpu:v25.8.0-dev`
4747
TEST_EXPECTED_IMAGE_SPEC_SUBSTRING ?= $(VERSION)
4848

4949
ifneq ($(TEST_CHART_LOCAL),"false")

0 commit comments

Comments
 (0)