Migrate H100/A100 runners to OSDC (ARC) by huydhn · Pull Request #179 · pytorch/pytorch-integration-testing

huydhn · 2026-05-29T17:35:55Z

Migrates this repo off the AWS H100/A100 runners onto OSDC (ARC) runners. OSDC/ARC runners are ephemeral pods with no Docker daemon, so the workflows that actually run on the migrated runners are switched from the docker run … docker exec pattern to a job-level container: (the GPU is injected by the runner pod). This matches pytorch/pytorch _linux-test.yml (test-osdc) and pytorch/helion.

1. Runner label migration (arc.yaml mapping + `mt-` prefix)

Old	OSDC label
`linux.aws.a100`	`mt-l-x86iavx512-11-125-a100`
`linux.aws.h100`	`mt-l-x86iamx-22-225-h100`
`linux.aws.h100.4`	`mt-l-x86iamx-88-900-h100-4`
`linux.aws.h100.8`	`mt-l-bx86iamx-176-1800-h100-8`

generate_vllm_benchmark_matrix.py gets the full label rename (TP_TO_RUNNER_MAPPING, RUNNER_TO_PLATFORM_MAPPING; the PLATFORM_SKIPS tokens become the bare h100/a100 so the substring matcher still skips the whole family). The matrix output is unchanged except for the runner strings. The test fixture is regenerated from the current benchmark configs (this also clears some pre-existing config drift).

2. Containerize the workflows that run on the migrated runners

vllm-ci-test.yml / vllm-profiling.yml — single-entry CUDA matrices that run only on the migrated mt- runners. Add an ubuntu-latest resolve-image pre-job (docker manifest inspect needs a daemon the pod lacks) to pick the latest available vLLM CI image and pass it to a container: (--gpus all). Drop the docker run/docker exec wrapper + /tmp/workspace bind mount; run scripts natively. run_vllm_profiling.sh now uses $GITHUB_WORKSPACE. Profiling assumes the upload IAM role via OIDC (no host instance role inside a pod).
pytorch-bisect.yaml — the mt- runner option builds PyTorch inside pytorch/pytorch:2.12.0-cuda13.0-cudnn9-devel via a conditional container:; linux.dgx.b200 keeps its bare-host path. CUDA_HOME points at the image CUDA on the container path.

Why the benchmark workflows are NOT touched

vllm-benchmark.yml / sglang-benchmark.yml in this repo do not execute on the migrated H100/A100 runners — those CUDA benchmarks run in pytorch/pytorch's vllm-benchmark.yml, which consumes the runner labels produced by generate_vllm_benchmark_matrix.py. The benchmark jobs here run on b200 / rocm / cpu / hpu, all of which keep a Docker daemon, so they keep the existing docker run pattern unchanged. (An earlier revision of this PR containerized them for all devices; that was reverted as out of scope — see 6f0e908.)

⚠️ Needs CI validation (no OSDC/host runner available locally)

Verified for YAML validity and the matrix test passes locally, but runtime behavior must be confirmed on real runners:

The bisect CUDA-devel image tag (2.12.0-cuda13.0-cudnn9-devel) and tritonparse build scripts working in-container.
OIDC token retrieval from inside the container for the profiling S3 upload role.
The resolved vLLM CI image running correctly as a job-level container: on the mt- pods.

Out of scope (not OSDC)

flash_attention.yml (b200 DGX), inductor.yml / tritonbench*.yml (AWS g5 / b200), and this repo's vllm-benchmark.yml / sglang-benchmark.yml (b200 / rocm / cpu / hpu) are left untouched.

🤖 Generated with Claude Code

Switch all linux.aws.h100* and linux.aws.a100 runner labels to their OSDC/ARC equivalents. Labels follow the mapping in pytorch/pytorch .github/arc.yaml, with the mt- (Meta multi-tenant) prefix that OSDC production runners use: linux.aws.a100 -> mt-l-x86iavx512-11-125-a100 (1 GPU) linux.aws.h100 -> mt-l-x86iamx-22-225-h100 (1 GPU) linux.aws.h100.4 -> mt-l-x86iamx-88-900-h100-4 (4 GPU) linux.aws.h100.8 -> mt-l-bx86iamx-176-1800-h100-8 (8 GPU) Files: - generate_vllm_benchmark_matrix.py: TP_TO_RUNNER_MAPPING and RUNNER_TO_PLATFORM_MAPPING get the full label rename. In PLATFORM_SKIPS the skip tokens become the bare GPU-type 'h100'/'a100' so they remain a substring of the OSDC names, preserving the 'skip the whole family' behavior the substring matcher relies on (matters for h100, which has 1/4/8-GPU variants). - vllm-ci-test.yml, vllm-profiling.yml, pytorch-bisect.yaml: runs-on / runner choice updated. - test fixture: expected runner values updated to the OSDC names. The matrix output is unchanged except for the runner label strings (verified: every model<->runner pairing is identical after the rename).

OSDC/ARC runners are ephemeral pods with no Docker daemon, so the old 'docker run --gpus all + docker exec' pattern cannot work on them. Run the vLLM CI image via the job-level container: key with options '--gpus all' instead (the GPU is injected by the runner pod), matching pytorch/pytorch _linux-test.yml (test-osdc) and pytorch/helion. - vllm-ci-test.yml / vllm-profiling.yml: add an ubuntu-latest 'resolve-image' pre-job that runs 'docker manifest inspect' (needs a daemon the pod lacks) to pick the latest available vLLM CI image and pass it down as the container image. Drop the GPU_FLAG/docker run/docker exec wrapper and the /tmp/workspace bind-mount; run the scripts directly in the container. - vllm-profiling.yml: assume the upload IAM role via OIDC before the S3 upload (ephemeral pods have no host instance role); pass the resolved vLLM commit through as S3_HEAD_SHA. - run_vllm_profiling.sh: use $GITHUB_WORKSPACE instead of the hardcoded /tmp/workspace bind-mount path.

@@ -52,11 +44,12 @@ jobs:
          ref: ${{ inputs.vllm_branch || 'main' }}
          fetch-depth: 0

-      - name: Set Docker registry
-        shell: bash
+      - name: Resolve the latest available vLLM CI image
+        id: resolve
+        working-directory: vllm
        env:
          HEAD_BRANCH: ${{ inputs.vllm_branch || 'main' }}
-          DEVICE_NAME: ${{ matrix.device-name }}
+          HEAD_SHA: ${{ inputs.vllm_commit || '' }}
        run: |
          set -eux

@@ -67,67 +60,59 @@ jobs:
            DOCKER_IMAGE_PREFIX=public.ecr.aws/q9t5s3a7/vllm-ci-test-repo
          fi

-          DOCKER_IMAGE_SUFFIX=""
-          if [[ "${DEVICE_NAME}" == "rocm" ]]; then
-            DOCKER_IMAGE_PREFIX=docker.io/rocm/vllm-ci
-          elif [[ "${DEVICE_NAME}" == "cpu" ]]; then
-            DOCKER_IMAGE_SUFFIX=-cpu
-          fi
-          echo "DOCKER_IMAGE_PREFIX=$DOCKER_IMAGE_PREFIX" >> $GITHUB_ENV
-          echo "DOCKER_IMAGE_SUFFIX=$DOCKER_IMAGE_SUFFIX" >> $GITHUB_ENV
-
-      - name: Check for available Docker image
-        working-directory: vllm
-        env:
-          HEAD_BRANCH: ${{ inputs.vllm_branch || 'main' }}
-          HEAD_SHA: ${{ inputs.vllm_commit || '' }}
-        run: |
-          set -eux
-
          if [[ -z "${HEAD_SHA}" ]]; then
            # Looking back the latest 100 commits is enough
-            for i in {0..99}
-            do
+            for i in {0..99}; do
              # Check if the image is there, if it doesn't then check an older one
              # because the commit is too recent
              HEAD_SHA=$(git rev-parse --verify HEAD~${i})
-              DOCKER_IMAGE="${DOCKER_IMAGE_PREFIX}:${HEAD_SHA}${DOCKER_IMAGE_SUFFIX}"
-
-              # No Docker image available yet because the commit is too recent
+              DOCKER_IMAGE="${DOCKER_IMAGE_PREFIX}:${HEAD_SHA}"
              if docker manifest inspect "${DOCKER_IMAGE}"; then
                break
              fi
            done
          fi

-          echo "HEAD_SHA=$HEAD_SHA" >> $GITHUB_ENV
+          echo "docker-image=${DOCKER_IMAGE_PREFIX}:${HEAD_SHA}" >> "${GITHUB_OUTPUT}"

-      - name: Setup CUDA GPU_FLAG for docker run
-        if: matrix.device-name == 'cuda'
+  test:
+    name: Run vLLM tests
+    needs: resolve-image
+    if: ${{ !github.event.pull_request.head.repo.fork && github.repository_owner == 'pytorch' }}
+    strategy:


@@ -124,98 +60,86 @@ jobs:
            DOCKER_IMAGE_PREFIX=public.ecr.aws/q9t5s3a7/vllm-ci-test-repo
          fi

-          DOCKER_IMAGE_SUFFIX=""
-          if [[ "${DEVICE_NAME}" == "rocm" ]]; then
-            DOCKER_IMAGE_PREFIX=docker.io/rocm/vllm-ci
-          elif [[ "${DEVICE_NAME}" == "cpu" ]]; then
-            DOCKER_IMAGE_SUFFIX=-cpu
-          fi
-          echo "DOCKER_IMAGE_PREFIX=$DOCKER_IMAGE_PREFIX" >> $GITHUB_ENV
-          echo "DOCKER_IMAGE_SUFFIX=$DOCKER_IMAGE_SUFFIX" >> $GITHUB_ENV
-
-      - name: Check for last commit
-        working-directory: vllm-profiling/vllm
-        env:
-          HEAD_BRANCH: ${{ inputs.vllm_branch || 'main' }}
-          HEAD_SHA: ${{ inputs.vllm_commit || '' }}
-        run: |
-          set -eux
-
          if [[ -z "${HEAD_SHA}" ]]; then
-            for i in {0..99}
-            do
+            for i in {0..99}; do
              HEAD_SHA=$(git rev-parse --verify HEAD~${i})
-              DOCKER_IMAGE="${DOCKER_IMAGE_PREFIX}:${HEAD_SHA}${DOCKER_IMAGE_SUFFIX}"
-
+              DOCKER_IMAGE="${DOCKER_IMAGE_PREFIX}:${HEAD_SHA}"
              # Docker image available for this commit, then exit
              if docker manifest inspect "${DOCKER_IMAGE}"; then
                break
              fi
            done
          fi

-          echo "HEAD_SHA=$HEAD_SHA" >> $GITHUB_ENV
+          echo "docker-image=${DOCKER_IMAGE_PREFIX}:${HEAD_SHA}" >> "${GITHUB_OUTPUT}"
+          echo "head-sha=${HEAD_SHA}" >> "${GITHUB_OUTPUT}"
          echo "### Run profiling on [${HEAD_SHA}](https://github.com/vllm-project/vllm/commit/${HEAD_SHA})" >> "${GITHUB_STEP_SUMMARY}"

-      - name: Setup CUDA GPU_FLAG for docker run
-        if: env.DEVICE_NAME == 'cuda'
+  profiling:
+    name: Run vLLM profiling
+    needs: resolve-image


…ainer The mt- runner is an ephemeral OSDC pod with no host CUDA toolchain, so build PyTorch inside pytorch/pytorch:2.12.0-cuda13.0-cudnn9-devel (--gpus all) when the mt- runner is selected; linux.dgx.b200 keeps the existing bare-host path (conditional container via fromJSON('null')). CUDA_HOME points at the image's /usr/local/cuda on the container path (run.sh requires it non-empty). Add a git safe.directory step for the root-owned in-container checkout.

Run every matrix device inside a job-level container: instead of the old 'docker run + docker exec' pattern, since OSDC/ARC pods have no Docker daemon. - generate_vllm_benchmark_matrix.py: emit a per-entry 'device-name' so the workflow can resolve the container image up front (regenerated the test fixture, which also clears pre-existing config drift). - set-parameters: resolve the upstream image on ubuntu-latest (which has a daemon) via 'docker manifest inspect', then enrich every matrix entry with container-image + device-appropriate container-options. sglang resolves per image suffix (cuda / -cu128-b200 / -rocm630-mi30x) and skips non-cuda/rocm devices instead of failing the whole matrix. - benchmarks job: add container: { image, options }, drop the device probe (device-name comes from the matrix) while keeping the runtime DEVICE_TYPE detection, run the benchmark script natively, and assume the upload IAM role via OIDC for all devices (no host instance role inside a pod). chown is made sudo-optional for the in-container root user. Flagged for CI validation: the per-model S3 'already benchmarked' dedup is dropped (needs the runtime device-type before the container exists); the rocm/hpu container options are best-effort; and --shm-size may be capped on ARC pods.

The H100/A100 benchmark jobs do not actually run in this repo - they run in pytorch/pytorch's .github/workflows/vllm-benchmark.yml, which consumes the runner labels emitted by generate_vllm_benchmark_matrix.py. So this repo's vllm-benchmark.yml / sglang-benchmark.yml never execute on the migrated OSDC runners and do not need the job-level container rewrite. Revert f6bc62f's four-file change: - vllm-benchmark.yml / sglang-benchmark.yml: back to the docker run/exec pattern (they run on b200 / rocm / cpu / hpu, all of which keep a Docker daemon - not OSDC). - generate_vllm_benchmark_matrix.py: drop the per-entry device-name field (it only existed to drive the dropped containerization). The H100/A100 -> OSDC label rename is kept, since pytorch/pytorch needs it. - test fixture: regenerated from the current benchmark configs (clears pre-existing config drift) with the mt- labels and no device-name. What remains is the minimal migration: the runner label rename plus the containerization of the three workflows that DO run on the migrated runners here (vllm-ci-test, vllm-profiling, pytorch-bisect).

huydhn had a problem deploying to pytorch-x-vllm May 29, 2026 17:36 — with GitHub Actions Error

meta-cla Bot added the cla signed label May 29, 2026

huydhn force-pushed the migrate-h100-to-osdc-runners branch from 8d498d9 to a7a54a0 Compare May 29, 2026 19:18

huydhn had a problem deploying to pytorch-x-vllm May 29, 2026 19:19 — with GitHub Actions Failure

huydhn changed the title ~~Migrate linux.aws.h100 runners to OSDC (ARC) runners~~ Migrate linux.aws.h100/a100 runners to OSDC (ARC) runners May 29, 2026

huydhn had a problem deploying to pytorch-x-vllm May 29, 2026 19:25 — with GitHub Actions Failure

huydhn had a problem deploying to pytorch-x-vllm May 29, 2026 19:50 — with GitHub Actions Failure

huydhn temporarily deployed to pytorch-x-vllm May 29, 2026 19:50 — with GitHub Actions Inactive

github-advanced-security Bot found potential problems May 29, 2026

View reviewed changes

huydhn had a problem deploying to pytorch-x-vllm May 29, 2026 20:05 — with GitHub Actions Failure

huydhn temporarily deployed to pytorch-x-vllm May 29, 2026 20:06 — with GitHub Actions Inactive

huydhn changed the title ~~Migrate linux.aws.h100/a100 runners to OSDC (ARC) runners~~ Migrate to OSDC runners and containerize the GPU workflows May 29, 2026

huydhn temporarily deployed to pytorch-x-vllm May 29, 2026 20:33 — with GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm May 29, 2026 20:33 — with GitHub Actions Failure

huydhn force-pushed the migrate-h100-to-osdc-runners branch from 5be1449 to f6bc62f Compare June 12, 2026 21:17

huydhn had a problem deploying to pytorch-x-vllm June 12, 2026 21:18 — with GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm June 12, 2026 21:18 — with GitHub Actions Inactive

huydhn changed the title ~~Migrate to OSDC runners and containerize the GPU workflows~~ Migrate H100/A100 runners to OSDC (ARC) Jun 12, 2026

huydhn temporarily deployed to pytorch-x-vllm June 12, 2026 21:33 — with GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm June 13, 2026 00:44 — with GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm June 13, 2026 00:44 — with GitHub Actions Failure

huydhn had a problem deploying to pytorch-x-vllm June 13, 2026 00:44 — with GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm June 13, 2026 00:44 — with GitHub Actions Failure

huydhn merged commit a51fced into main Jun 13, 2026
8 of 17 checks passed

Navigation Menu

Uh oh!

Migrate H100/A100 runners to OSDC (ARC) - #179

Migrate H100/A100 runners to OSDC (ARC)#179
huydhn merged 5 commits into
mainfrom
migrate-h100-to-osdc-runners

huydhn commented May 29, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

huydhn commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. Runner label migration (arc.yaml mapping + mt- prefix)

2. Containerize the workflows that run on the migrated runners

Why the benchmark workflows are NOT touched

⚠️ Needs CI validation (no OSDC/host runner available locally)

Out of scope (not OSDC)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

huydhn commented May 29, 2026 •

edited

Loading

1. Runner label migration (arc.yaml mapping + `mt-` prefix)