Skip to content

Functional Tests - Training (Megatron-LM-FL) #312

Functional Tests - Training (Megatron-LM-FL)

Functional Tests - Training (Megatron-LM-FL) #312

Manually triggered June 26, 2026 06:51
Status Failure
Total duration 1d 11h 1m 42s
Artifacts 6
checkout_and_config
7s
checkout_and_config
Matrix: functional_test_hetero_train
Matrix: functional_test_train
Fit to window
Zoom out
Zoom in

Annotations

12 errors and 27 warnings
functional_test_hetero_train (a100, hetero_train, aquila, dp2dp4_shared_embedding)
Process completed with exit code 1.
functional_test_train (a100, train, aquila, tp2_pp2)
Process completed with exit code 1.
functional_test_hetero_train (a100, hetero_train, aquila, tp2pp1_tp4pp1_tp2pp1)
Failed to download archive 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02' after 3 attempts.
functional_test_hetero_train (a100, hetero_train, aquila, tp2pp1_tp4pp1_tp2pp1)
Action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02' download has timed out. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, aquila, tp4_pp2)
Process completed with exit code 1.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2_engram)
Failed to download archive 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02' after 3 attempts.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2_engram)
Action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02' download has timed out. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, mixtral, tp2_pp1_ep2)
Process completed with exit code 1.
functional_test_train (a100, train, mixtral, tp4_pp1_ep2)
Process completed with exit code 1.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2)
Process completed with exit code 1.
functional_test_hetero_train (a100, hetero_train, aquila, tp2dp1pp1_tp2dp2pp1_tp1dp2pp1)
The job was not acquired by Runner of type scale-set even after multiple attempts
Functional Tests - Training (Megatron-LM-FL)
The job was not acquired by Runner of type scale-set even after multiple attempts
checkout_and_config
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_hetero_train (a100, hetero_train, aquila, dp2dp4_shared_embedding)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_train (a100, train, aquila, tp2_pp2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_train (a100, train, aquila, tp2_pp2)
Back off 15.749 seconds before retry.
functional_test_train (a100, train, aquila, tp2_pp2)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, aquila, tp2_pp2)
Back off 13.891 seconds before retry.
functional_test_train (a100, train, aquila, tp2_pp2)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_hetero_train (a100, hetero_train, aquila, tp2pp1_tp4pp1_tp2pp1)
Back off 19.251 seconds before retry.
functional_test_hetero_train (a100, hetero_train, aquila, tp2pp1_tp4pp1_tp2pp1)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_hetero_train (a100, hetero_train, aquila, tp2pp1_tp4pp1_tp2pp1)
Back off 18.826 seconds before retry.
functional_test_hetero_train (a100, hetero_train, aquila, tp2pp1_tp4pp1_tp2pp1)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, aquila, tp4_pp2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_train (a100, train, deepseek, tp2_pp2_ep2_engram)
Back off 22.316 seconds before retry.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2_engram)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2_engram)
Back off 11.431 seconds before retry.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2_engram)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, mixtral, tp2_pp1_ep2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_train (a100, train, mixtral, tp2_pp1_ep2)
Back off 16.432 seconds before retry.
functional_test_train (a100, train, mixtral, tp2_pp1_ep2)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, mixtral, tp4_pp1_ep2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_train (a100, train, mixtral, tp4_pp1_ep2)
Back off 17.978 seconds before retry.
functional_test_train (a100, train, mixtral, tp4_pp1_ep2)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, mixtral, tp4_pp1_ep2)
Back off 17.239 seconds before retry.
functional_test_train (a100, train, mixtral, tp4_pp1_ep2)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2)
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
functional_test_train (a100, train, deepseek, tp2_pp2_ep2)
Back off 26.616 seconds before retry.
functional_test_train (a100, train, deepseek, tp2_pp2_ep2)
Failed to download action 'https://codeload.github.com/actions/upload-artifact/tar.gz/ea165f8d65b6e75b540449e92b4886f43607fa02'. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

Artifacts

Produced during runtime
Name Size Digest
functional_tests-hetero-megatron-lm-fl-logs-28222278385-hetero_train-aquila Expired
11.8 KB
sha256:ee5eb5ed83bc76d83b7fb6cdc68a6004678e05675811e7402c508bce2da7d0f1
functional_tests-megatron-lm-fl-logs-28222278385-train-aquila Expired
13.9 KB
sha256:1c58acc5cdd7bcf0166c5ae485cc6b482818cb29c2d2465c91e56074082ba8d2
functional_tests-megatron-lm-fl-logs-28222278385-train-aquila Expired
13.9 KB
sha256:d12e2e69d6aad85b06bb257f34f70756dc46aaf8e96fa1050953984b1490f2b4
functional_tests-megatron-lm-fl-logs-28222278385-train-deepseek Expired
14.3 KB
sha256:b5861842145da638c0a1003e174a64de8218802eb9bfac6e2ce3c6e82175fa74
functional_tests-megatron-lm-fl-logs-28222278385-train-mixtral Expired
14 KB
sha256:85796bc5c4ff7ed685704ed2e485eecbce5ed50f1cbf3d7fd104214b35346822
functional_tests-megatron-lm-fl-logs-28222278385-train-mixtral Expired
14 KB
sha256:5831e3586abf03f08848535d0f7e93d686287f9d6d0cb45b877778acbed25396