Commit 13f4e3b
committed
Exclude CUDA 13.4 from the docker release/validation matrix
**Impact:** CI only — the nightly/test "Build Official Docker Images" validation matrix in pytorch/pytorch (no published images change)
**Risk:** low
## What
Adds a `CUDA_ARCHES_NO_DOCKER = ["13.4"]` exclusion list and filters it out of the docker matrix in `generate_docker_release_matrix.py`, so the docker validation matrix stops asking for CUDA 13.4 images. Wheel, libtorch and domain-library matrices are untouched.
## Why
The nightly `Build Official Docker Images` workflow has been red every night since 2026-08-11 — the `validate` stage tries to `docker pull` `cuda13.4-cudnn9-{runtime,devel}` images that the `build` stage in the same run never produces, failing with `manifest unknown` (e.g. [run 32626369633, cuda13.4-cudnn9-runtime](https://github.com/pytorch/pytorch/actions/runs/32626369633/job/97164375717)).
The root cause is drift between two independently-maintained CUDA lists in two repos. The build matrix comes from `CUDA_ARCHES_FULL_VERSION` in pytorch/pytorch, which deliberately withholds 13.4 (its release Dockerfile installs the devel toolkit from the standard NVIDIA apt repo, but `cuda-toolkit-13-4` ships only in NVIDIA's *preview* channel so far). The validate matrix comes from `CUDA_ARCHES_DICT` in test-infra, which grew a 13.4 entry (test-infra #8477) so torchvision/torchaudio would build cu134 — and that same constant, unknowingly, also feeds pytorch/pytorch's docker validation via the `@main`-pinned reusable workflow. Adding 13.4 for domain libraries silently turned on validation of docker images that were never meant to be built.
The fix keeps pytorch/pytorch's deliberate exclusion intact and re-aligns the validation matrix on the test-infra side, where the drift was introduced. No real coverage is lost — no cuda13.4 image exists to validate. The exclusion is applied across all channels, which also closes the same latent failure on the `test` channel before it gets hit.
## Notes
Full root-cause writeup, timeline, and options considered are in the analysis doc this change is based on. When revisiting: once NVIDIA promotes CUDA 13.4 into the standard apt repo, the images can be built (a two-line change to `CUDA_ARCHES_FULL_VERSION` in pytorch/pytorch) and `CUDA_ARCHES_NO_DOCKER` can drop back to `[]`.
Known gap this doesn't address: nothing tests `generate_docker_release_matrix.py`, and the validate job reports a bare `manifest unknown` rather than "image never built" — which is why this drift went unexamined for ~two weeks. Both are worth a follow-up but are out of scope here.
Authored with assistance from an AI coding assistant (Claude).
Signed-off-by: Jean Schmidt <contato@jschmidt.me>1 parent 61a7757 commit 13f4e3b
2 files changed
Lines changed: 14 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
87 | 95 | | |
88 | 96 | | |
89 | 97 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
48 | 53 | | |
49 | 54 | | |
50 | 55 | | |
| |||
0 commit comments