Skip to content

Commit 4c1abab

Browse files
zhmiaoCopilot
andcommitted
ci(release): RP-5 — add Docker build + push to GHCR; gate GH Release publish
Adds 2 new release.yml jobs on prod tag push: - build-and-push-docker-cpu (needs publish-pypi-cpu) — pushes ghcr.io/microsoft/sparrow-engine-server:vX.Y.Z + :sparrow-combined - build-and-push-docker-gpu (needs publish-pypi-gpu) — pushes ghcr.io/microsoft/sparrow-engine-server-gpu:vX.Y.Z + :sparrow-combined Modifies publish-cli-release-assets: - Adds both docker jobs to needs: - Adds both to the if: success-check (failure of either now blocks GH Release publish) Why gate the GH Release publish on docker push success: The sparrow webapp consumes a pinned engine image. RP-5 wires an auto-PR on the sparrow repo (Clamps251/sparrow) that polls microsoft/Pytorch-Wildlife/releases/latest every 30 min. Gating the GH Release publish on docker push makes 'release published' a strict happens-after of 'both images on GHCR' — sparrow's poll is race-free. Tradeoff: docker failure blocks the GH Release page (PyPI wheels still publish earlier in the chain). Acceptable; failures are rare and fixing the Dockerfile unblocks both via workflow_dispatch. Runner: free ubuntu-latest works for the GPU image despite no GPU present (cudarc fallback-dynamic-loading + nvjpeg-sys pre-generated bindings skip CUDA Toolkit at build time — RP-19 precedent for the Windows GPU wheel). Disk: free-disk-space action pre-step on the GPU job to ensure 6-8GB headroom for the cuDNN base layer. Cache: separate ghcr.io/microsoft/sparrow-engine-server-buildcache:cpu (and :gpu) registry caches so runtime image tag lists stay clean (the sparrow poll workflow filters tags by shape; a 'buildcache' tag would confuse downstream introspection). GHCR auth via GITHUB_TOKEN + packages: write — zero secrets to manage. Image visibility flips to PUBLIC manually after first push (one-time operator action; documented in RP-5 final_design.md § Test plan Phase 1). Sparrow-side workflow + baseline_up.sh + pin contract changes land separately on Clamps251/sparrow @ feat/rp-5-image-pin-auto-pr. Refs: - Design: sparrow-engine-dev:docs/design/rp-5-image-pin-auto-pr/final_design.md - Ledger: rp-5-implement (sparrow-engine-dev session 4f3f88eb) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent b710048 commit 4c1abab

1 file changed

Lines changed: 105 additions & 0 deletions

File tree

.github/workflows/release.yml

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1162,6 +1162,107 @@ jobs:
11621162
sparrow-engine/dist/sparrow-engine-cpu-*-windows-x86_64.zip.sha256
11631163
if-no-files-found: error
11641164

1165+
# ---------------------------------------------------------------------------
1166+
# RP-5 — Build + push CPU/GPU Docker server images to GHCR on prod tag push.
1167+
# Hyphenated tags (RC / pre) skip (same shape as publish-pypi-*). Failure of
1168+
# either job blocks the GitHub Release publish below (so sparrow's poll on
1169+
# /releases/latest is race-free — see RP-5 final_design.md). Free
1170+
# ubuntu-latest runner suffices for both: cudarc fallback-dynamic-loading +
1171+
# nvjpeg-sys pre-generated bindings skip the build-time CUDA Toolkit
1172+
# requirement (RP-19 precedent for the Windows GPU wheel).
1173+
# ---------------------------------------------------------------------------
1174+
1175+
build-and-push-docker-cpu:
1176+
name: Build + push CPU Docker image to GHCR
1177+
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, '-')
1178+
needs: [publish-pypi-cpu]
1179+
runs-on: ubuntu-latest
1180+
permissions:
1181+
contents: read
1182+
packages: write
1183+
outputs:
1184+
digest: ${{ steps.push.outputs.digest }}
1185+
steps:
1186+
- uses: actions/checkout@v4
1187+
1188+
- uses: docker/setup-buildx-action@v3
1189+
1190+
- uses: docker/login-action@v3
1191+
with:
1192+
registry: ghcr.io
1193+
username: ${{ github.actor }}
1194+
password: ${{ secrets.GITHUB_TOKEN }}
1195+
1196+
- id: push
1197+
uses: docker/build-push-action@v5
1198+
with:
1199+
context: sparrow-engine
1200+
file: sparrow-engine/docker/Dockerfile.cpu
1201+
push: true
1202+
tags: |
1203+
ghcr.io/microsoft/sparrow-engine-server:${{ github.ref_name }}
1204+
ghcr.io/microsoft/sparrow-engine-server:sparrow-combined
1205+
labels: |
1206+
org.opencontainers.image.source=https://github.com/microsoft/Pytorch-Wildlife
1207+
org.opencontainers.image.revision=${{ github.sha }}
1208+
org.opencontainers.image.version=${{ github.ref_name }}
1209+
# Buildcache lives in a SEPARATE image (not as a tag on the runtime
1210+
# image) so the runtime image's tag list stays clean — sparrow's
1211+
# auto-PR workflow filters tags by shape, and a `buildcache` tag
1212+
# would confuse downstream introspection. Cache stays private to PW.
1213+
cache-from: type=registry,ref=ghcr.io/microsoft/sparrow-engine-server-buildcache:cpu
1214+
cache-to: type=registry,ref=ghcr.io/microsoft/sparrow-engine-server-buildcache:cpu,mode=max
1215+
1216+
build-and-push-docker-gpu:
1217+
name: Build + push GPU Docker image to GHCR
1218+
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, '-')
1219+
needs: [publish-pypi-gpu]
1220+
runs-on: ubuntu-latest
1221+
permissions:
1222+
contents: read
1223+
packages: write
1224+
outputs:
1225+
digest: ${{ steps.push.outputs.digest }}
1226+
steps:
1227+
- uses: actions/checkout@v4
1228+
1229+
- uses: docker/setup-buildx-action@v3
1230+
1231+
- uses: docker/login-action@v3
1232+
with:
1233+
registry: ghcr.io
1234+
username: ${{ github.actor }}
1235+
password: ${{ secrets.GITHUB_TOKEN }}
1236+
1237+
# Free 15GB on ubuntu-latest before the ~4GB cuDNN runtime base pull.
1238+
# Default runner has 14GB free disk; GPU image build needs ~6-8GB
1239+
# working space (2GB base + layers + buildkit cache mounts).
1240+
- name: Free runner disk space (cuDNN base is ~2GB)
1241+
uses: jlumbroso/free-disk-space@v1.3.1
1242+
with:
1243+
tool-cache: false # keep nodejs/python — buildx needs them
1244+
android: true
1245+
dotnet: true
1246+
haskell: true
1247+
large-packages: true
1248+
docker-images: false # we need docker
1249+
1250+
- id: push
1251+
uses: docker/build-push-action@v5
1252+
with:
1253+
context: sparrow-engine
1254+
file: sparrow-engine/docker/Dockerfile.gpu
1255+
push: true
1256+
tags: |
1257+
ghcr.io/microsoft/sparrow-engine-server-gpu:${{ github.ref_name }}
1258+
ghcr.io/microsoft/sparrow-engine-server-gpu:sparrow-combined
1259+
labels: |
1260+
org.opencontainers.image.source=https://github.com/microsoft/Pytorch-Wildlife
1261+
org.opencontainers.image.revision=${{ github.sha }}
1262+
org.opencontainers.image.version=${{ github.ref_name }}
1263+
cache-from: type=registry,ref=ghcr.io/microsoft/sparrow-engine-server-buildcache:gpu
1264+
cache-to: type=registry,ref=ghcr.io/microsoft/sparrow-engine-server-buildcache:gpu,mode=max
1265+
11651266
# ---------------------------------------------------------------------------
11661267
# Attach CLI tarballs as GitHub Release assets on prod tag push.
11671268
# Hyphenated tags (RC / pre) skip publish; build jobs still ran above.
@@ -1184,12 +1285,16 @@ jobs:
11841285
&& needs['build-cli-linux-gpu'].result == 'success'
11851286
&& needs['build-cli-macos-arm64'].result == 'success'
11861287
&& needs['server-boot-smoke'].result == 'success'
1288+
&& needs['build-and-push-docker-cpu'].result == 'success'
1289+
&& needs['build-and-push-docker-gpu'].result == 'success'
11871290
needs:
11881291
- build-cli-linux-cpu
11891292
- build-cli-linux-gpu
11901293
- build-cli-macos-arm64
11911294
- build-cli-windows
11921295
- server-boot-smoke
1296+
- build-and-push-docker-cpu
1297+
- build-and-push-docker-gpu
11931298
runs-on: ubuntu-latest
11941299
permissions:
11951300
# softprops/action-gh-release needs write access to create / append the

0 commit comments

Comments
 (0)