Skip to content

CI

CI #4986

Triggered via schedule October 26, 2025 09:30
Status Failure
Total duration 1h 16m 0s
Artifacts 46

ci.yaml

on: schedule
metadata
4s
metadata
bump-manifest
26s
bump-manifest
Matrix: amd64 / test-distribution
Matrix: arm64 / test-distribution
amd64  /  ...  /  build-base
2m 50s
amd64 / build-base / build-base
arm64  /  ...  /  build-base
3m 22s
arm64 / build-base / build-base
amd64  /  ...  /  build-mpi-operator-compatible-base
2m 15s
amd64 / test-nccl / build-mpi-operator-compatible-base
amd64  /  ...  /  build-nccl-gke
1m 53s
amd64 / test-nccl / nccl-test-gke / build-nccl-gke
arm64  /  ...  /  build-mpi-operator-compatible-base
arm64 / test-nccl / build-mpi-operator-compatible-base
arm64  /  ...  /  build-nccl-gke
arm64 / test-nccl / nccl-test-gke / build-nccl-gke
Matrix: amd64 / test-jax-cutlass-h100 / jax-cutlass-test-h100
Matrix: amd64 / test-jax / run-unit-test
Matrix: amd64 / test-te-a100 / run-unit-test
Matrix: amd64 / test-te-h100 / te-test-h100
amd64  /  ...  /  launch-slurm-runner
17m 42s
amd64 / test-jax / runner / launch-slurm-runner
amd64  /  test-nsys-jax-eks
3m 6s
amd64 / test-nsys-jax-eks
amd64  /  ...  /  launch-slurm-runner
22m 54s
amd64 / test-te-a100 / runner / launch-slurm-runner
amd64  /  build-upstream-t5x
6m 51s
amd64 / build-upstream-t5x
Matrix: amd64 / test-nsys-jax / run-unit-test
amd64  /  ...  /  launch-slurm-runner
12m 46s
amd64 / test-nsys-jax / runner / launch-slurm-runner
Matrix: amd64 / test-nccl / nccl-test
Matrix: amd64 / test-nccl / nccl-test-gke / nccl-gke
Matrix: arm64 / test-jax-cutlass-h100 / jax-cutlass-test-h100
Waiting for pending jobs
Matrix: arm64 / test-jax / run-unit-test
Waiting for pending jobs
Matrix: arm64 / test-te-a100 / run-unit-test
Waiting for pending jobs
Matrix: arm64 / test-te-h100 / te-test-h100
Waiting for pending jobs
arm64  /  test-nsys-jax-eks
0s
arm64 / test-nsys-jax-eks
arm64  /  ...  /  launch-slurm-runner
arm64 / test-jax / runner / launch-slurm-runner
arm64  /  ...  /  launch-slurm-runner
arm64 / test-te-a100 / runner / launch-slurm-runner
arm64  /  build-upstream-t5x
9m 58s
arm64 / build-upstream-t5x
Matrix: arm64 / test-nsys-jax / run-unit-test
Waiting for pending jobs
arm64  /  ...  /  launch-slurm-runner
arm64 / test-nsys-jax / runner / launch-slurm-runner
Matrix: arm64 / test-nccl / nccl-test
Waiting for pending jobs
Matrix: arm64 / test-nccl / nccl-test-gke / nccl-gke
Waiting for pending jobs
amd64  /  ...  /  maxtext-gke-xpk
2m 5s
amd64 / test-maxtext-gke / maxtext-gke-xpk
Matrix: amd64 / test-maxtext / maxtext-multinode
Matrix: amd64 / test-maxtext / single-process-multi-device
amd64  /  ...  /  build-rosetta
13m 54s
amd64 / build-rosetta-t5x / build-rosetta
amd64  /  test-axlearn-eks
2m 22s
amd64 / test-axlearn-eks
amd64  /  test-axlearn-fuji-models-eks
43s
amd64 / test-axlearn-fuji-models-eks
Matrix: amd64 / test-nsys-jax-archive
arm64  /  ...  /  maxtext-gke-xpk
arm64 / test-maxtext-gke / maxtext-gke-xpk
Matrix: arm64 / test-maxtext / maxtext-multinode
Waiting for pending jobs
Matrix: arm64 / test-maxtext / single-process-multi-device
Waiting for pending jobs
arm64  /  ...  /  build-rosetta
15m 51s
arm64 / build-rosetta-t5x / build-rosetta
arm64  /  test-axlearn-eks
0s
arm64 / test-axlearn-eks
arm64  /  test-axlearn-fuji-models-eks
0s
arm64 / test-axlearn-fuji-models-eks
Matrix: arm64 / test-nsys-jax-archive
amd64  /  ...  /  test-maxtext-metrics
14s
amd64 / test-maxtext / test-maxtext-metrics
amd64  /  collect-docker-tags
2s
amd64 / collect-docker-tags
Matrix: amd64 / test-rosetta-t5x / vit-multi-gpu-multi-node
arm64  /  ...  /  test-maxtext-metrics
arm64 / test-maxtext / test-maxtext-metrics
arm64  /  collect-docker-tags
3s
arm64 / collect-docker-tags
Matrix: arm64 / test-rosetta-t5x / vit-multi-gpu-multi-node
Waiting for pending jobs
amd64  /  ...  /  sitrep
6s
amd64 / test-maxtext / test-maxtext-sitrep / sitrep
amd64  /  ...  /  test-t5x-rosetta-summary
3s
amd64 / test-rosetta-t5x / test-t5x-rosetta-summary
amd64  /  ...  /  test-t5x-rosetta-metrics
18s
amd64 / test-rosetta-t5x / test-t5x-rosetta-metrics
arm64  /  ...  /  sitrep
arm64 / test-maxtext / test-maxtext-sitrep / sitrep
arm64  /  ...  /  test-t5x-rosetta-summary
arm64 / test-rosetta-t5x / test-t5x-rosetta-summary
arm64  /  ...  /  test-t5x-rosetta-metrics
arm64 / test-rosetta-t5x / test-t5x-rosetta-metrics
amd64  /  ...  /  test-maxtext-outcome
2s
amd64 / test-maxtext / test-maxtext-outcome
amd64  /  ...  /  sitrep
6s
amd64 / test-rosetta-t5x / test-t5x-rosetta-sitrep / sitrep
arm64  /  ...  /  test-maxtext-outcome
arm64 / test-maxtext / test-maxtext-outcome
arm64  /  ...  /  sitrep
arm64 / test-rosetta-t5x / test-t5x-rosetta-sitrep / sitrep
amd64  /  ...  /  test-t5x-rosetta-outcome
3s
amd64 / test-rosetta-t5x / test-t5x-rosetta-outcome
arm64  /  ...  /  test-t5x-rosetta-outcome
arm64 / test-rosetta-t5x / test-t5x-rosetta-outcome
make-publish-configs
4s
make-publish-configs
merge-new-manifest
7s
merge-new-manifest
Matrix: publish-containers
finalize  /  workflow-badge
7s
finalize / workflow-badge
finalize  /  report
9s
finalize / report
finalize  /  upload-badge
15s
finalize / upload-badge
finalize  /  publish-badge
5s
finalize / publish-badge
Fit to window
Zoom out
Zoom in

Annotations

11 errors and 2 warnings
amd64 / test-te-h100 / te-test-h100 (unittest, 8)
Process completed with exit code 1.
amd64 / test-jax / jax-A100-unit-test
Process completed with exit code 1.
amd64 / test-nsys-jax / nsys-jax-A100-unit-test
EACCES: permission denied, scandir '/runner/_work/JAX-Toolbox/JAX-Toolbox/pytest-tmp'
amd64 / test-nsys-jax / nsys-jax-A100-unit-test
Process completed with exit code 1.
amd64 / test-nsys-jax / nsys-jax-A100-unit-test
Process completed with exit code 1.
amd64 / test-te-a100 / te-A100-unit-test
Process completed with exit code 1.
amd64 / test-rosetta-t5x / test-t5x-rosetta-metrics
Process completed with exit code 1.
amd64 / test-rosetta-t5x / test-t5x-rosetta-outcome
Process completed with exit code 1.
amd64 / test-maxtext-gke / maxtext-gke-xpk
Process completed with exit code 1.
amd64 / test-maxtext / test-maxtext-metrics
Process completed with exit code 1.
amd64 / test-maxtext / test-maxtext-outcome
Process completed with exit code 1.
merge-new-manifest
Unexpected input(s) 'owner_and_repo', valid inputs are ['route', 'mediaType']
merge-new-manifest
Unexpected input(s) 'owner_and_repo', 'head', 'base', 'body', 'title', 'draft', valid inputs are ['route', 'mediaType']

Artifacts

Produced during runtime
Name Size Digest
artifact-axlearn-build-amd64
567 Bytes
sha256:fa599de5f19d41df603f9867010f03adab6d724049a0b1977e02b21f03875f2b
artifact-axlearn-build-arm64
567 Bytes
sha256:5a005a1b54deaae989e2ff6bd9f077781357c6db8aad9ce4f16d0e3f47c2f86d
artifact-axlearn-test
36 KB
sha256:54408ad72580a02a9bfbbc55347b67c9dd73c75f58c175b5922b6b66ccf47077
artifact-base-build-amd64
567 Bytes
sha256:a60745365d5d00927d373a19e57548ee9efdb4ab4636f2078ba55f336c4b0b5f
artifact-base-build-arm64
567 Bytes
sha256:18575103083ffc64916ba5f20d4c243296f8e2ec9a3767ec26ea9445a33954e4
artifact-equinox-build-amd64
570 Bytes
sha256:240e71fcd06543175407253d0c0e3df526850b297296f23fb9ea298eb0ae432a
artifact-equinox-build-arm64
568 Bytes
sha256:1b82772fe41db35680082adf3ae7eff1e48237c2d1941f683ca042931d2846f9
artifact-final-report
3.09 KB
sha256:8111268bb358b5cec9fc8903e83de12c70b5fbcb6ff1b3104231551a85bc3db5
artifact-jax-build-amd64
553 Bytes
sha256:3b6111f6cf856980b0f105f7aabba90792a5b1bae45b1fde33929419cd1d3247
artifact-jax-build-arm64
554 Bytes
sha256:b229533c9e4a04044ac07397e3ec170e616ec1da455f292a086b18395ba9d9a4
artifact-maxtext-build-amd64
568 Bytes
sha256:52c5f5b55ee3509c47f66e3c34420a19c586b6296f7d68da34cba88f3ad2933c
artifact-maxtext-build-arm64
568 Bytes
sha256:106cdc07451c120bc20a09e6a597fa7f68264997da31db931b02544b6467a754
artifact-maxtext-test
631 Bytes
sha256:95f387b38cb6f6ddef1f47bc7610e057d6bf92dc9f628f0ba9f2d063b2a9e6f8
artifact-mpi-operator-compatible-base-build-amd64
637 Bytes
sha256:5c2986d5d0b0c95be11d89e63ecb6ab381c7af6a8a1b0c3a7feabc4db1ec9981
artifact-nccl-gke-build-amd64
571 Bytes
sha256:58421e1bcf0c575b8da1e8da4b93e9c8c23d7dea9d46ed0cf22cd971d9fc9c01
artifact-rosetta-build-t5x-amd64
584 Bytes
sha256:a7c1c60d4d6ff336cd6bc4f3a55deda32397d1257316852df8bd3cb8a71f3cbe
artifact-rosetta-build-t5x-arm64
585 Bytes
sha256:2a833ca982ffae0ab3dcb532317375aeaa727454c07dea60b0bcf86859dde4a2
artifact-rosetta-t5x-mgmn-test
624 Bytes
sha256:9f7ee3be1d293a0e847eaef558a51e89799a914dd6ba72d2341822e18c6b7e6c
artifact-t5x-build-amd64
568 Bytes
sha256:a18084d3ad49baa509141074052c65ff76818b536193979bf7473fdb85577241
artifact-t5x-build-arm64
567 Bytes
sha256:ea747cbea5b5d7ee7d638e46918667e33e22393ab347e645b54f2d0fc8c775c4
artifact-workflow-metadata
278 Bytes
sha256:d89fab85849055e9e4d82a5b2ae5e8e0ccb8de70cb46eef6014a6456aae8f8bb
bumped-manifest
51.5 KB
sha256:448e1cc35c04ba7381cb5c644b199aac2f3eeb928e99b1a5c1d496fc63164cc8
final-axlearn
258 Bytes
sha256:64b3870bae42d6169902be0b1719af76e6b5b9e7d1aef9d8ef45930263a2dc73
final-base
249 Bytes
sha256:8f9c410d303e2174ba81d2b91c8ebb51c54b2f71d723d49191eaff1c0835dc48
final-equinox
258 Bytes
sha256:80ec100723578232211e94db0c98aaacc803bdb8f695f00fbcce1c079cc4babe
final-jax
246 Bytes
sha256:175e9a585e30d4550cfdaa07a3a8cd21366c19139f6ab6e37db86dee2813bd4c
final-maxtext
258 Bytes
sha256:61504dddf3d4957945fd1a1377542dbe37db0631f87c0d71857a6e89dbe997bd
final-t5x
246 Bytes
sha256:d26e904de5efded7a3612694b4902377f4c94d92f9c9ec786fbe4961dcde675f
final-upstream-t5x
272 Bytes
sha256:ba51ad682e6e0ed2bd3b8b218daeb495f369af286c352f2b687e21824329d0b7
jax-cutlass-test-H100
1.24 KB
sha256:b2ebe81d8ec01dc31f1c8744e146ff1ff6ab76be74da50b6e48bec7f9bafa5db
jax-unit-test-A100
86.8 KB
sha256:de54519e7fe046476b9111b6515702377349b4a374a2259ec99ca5e9fa8162ad
mealkit-axlearn
268 Bytes
sha256:dc1777e6ee3965c40205ba872c83da38637e8027da75a91f19923e938dcb1216
mealkit-equinox
269 Bytes
sha256:1507d31d2aecd2b84a744fbd03715bcdc4291233adec88a092806129d6412f37
mealkit-jax
256 Bytes
sha256:abb1c6e695455da6bbfba3c1a158df299701ee4a996aeee8655a3fbfc2a3c5d4
mealkit-maxtext
268 Bytes
sha256:169a8efd3211bbaa2c08d2fa597d1f8bf3c0553ae231df9d58385b8ca191b3e4
mealkit-t5x
257 Bytes
sha256:4f711a1d13a6e29920a3a0a4211e3e51799e33fdfe00ab50db4fe6151450fd42
mealkit-upstream-t5x
281 Bytes
sha256:97cdb3b81c65b08ead018d4e13d53efbe82c6f5679efcdf96608f2721dfaf3b1
nccl-gke-all-gather
15.4 KB
sha256:884da6112dc9613aff159304d6ea994f92c151ea297b63a22089c1df943a513b
nccl-gke-all-reduce
15.5 KB
sha256:a64ca883d2d975b8de11b547b1f2fb1d5acd37f7ad1aca955471677c051eb4db
nccl-gke-broadcast
15.3 KB
sha256:40e8de2232e891dda29f4752aca2ef51707ff10483305a86c45eb7fdda60c1f3
nccl-gke-reduce-scatter
12.6 KB
sha256:c5df4c4d4c7c96afe6242468e170e5030d8c66be858786e9018ed2d8e7a751ff
rosetta-t5x-vit-18816063817-VIT8G1N
2.3 KB
sha256:a8a3cf60b7d049327c8dc4d1bdd672f4ab7e4fc4e3c3d49f8c09e26b4ddbe152
te-unit-test-A100
13.1 KB
sha256:2eba281480eecb0777285ef690c4f1afdc46d59e6fd3524cbf758ee324a58393
te-unit-test-H100
10.3 KB
sha256:f98875c5a4e5b0cad1ffe1a9e6964a9e84d1ffbecd690a640c48d1baae545305
upstream-maxtext-18816063817-1DP2FSDP4TP1PP_single_process
2.89 KB
sha256:aae0da2a62db956ce3187e5831d678391a6f0f60b5f5bab3747fd16597e05e9e
upstream-maxtext-18816063817-2DP2FSDP2TP1PP
4.54 KB
sha256:04cb11a2339073a41ac2557064793b653074f040b78cff6c2ad5870463be9d75