Skip to content

Commit 30f6c3d

Browse files
authored
Add migrate-workflow-ec2-to-osdc Claude Code skill (#8022)
Playbook for migrating pytorch/pytorch .github/workflows/*.yml from EC2 to OSDC (ARC) runners. Covers both dial-up (use-arc gated by the runner determinator) and 100% opt-in patterns, and documents the four inputs (use-arc, python-version, compiler, cuda-version) that must be plumbed through _linux-build.yml / _linux-test.yml so the build-osdc / test-osdc code paths activate.
1 parent f58aac0 commit 30f6c3d

1 file changed

Lines changed: 122 additions & 0 deletions

File tree

  • .claude/skills/migrate-workflow-ec2-to-osdc
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
---
2+
name: migrate-workflow-ec2-to-osdc
3+
description: Step-by-step playbook for migrating a pytorch/pytorch .github/workflows/*.yml from EC2 to OSDC (ARC) runners — covers both dial-up and 100% opt-in patterns, with the inputs that must be plumbed through _linux-build.yml / _linux-test.yml. Use when migrating a workflow off EC2 onto on-site data center / EKS-hosted self-hosted runners (OSDC / ARC), enabling the ARC experiment, or wiring up `use-arc` / `runner_prefix` / `python-version` / `compiler` / `cuda-version` inputs.
4+
---
5+
6+
# Migrating a workflow from EC2 to OSDC
7+
8+
OSDC (= ARC = on-site data center, EKS-hosted self-hosted runners) replaces EC2 runners. Migration touches the workflow file *and* requires a few inputs to flow into the reusable `_linux-build.yml` / `_linux-test.yml` so the OSDC code paths (`build-osdc`, `test-osdc`) activate. EC2 paths (`build`, `test`) and OSDC paths gate on `inputs.use-arc` (`!inputs.use-arc` vs `inputs.use-arc`), so flipping that input swaps execution lanes.
9+
10+
This playbook is for workflows that **call the reusable `_linux-build.yml` / `_linux-test.yml`** (e.g. `test-b200.yml`, `pull.yml`, `trunk.yml`, `tsan.yml`, `operator_microbenchmark.yml`). For *standalone* OSDC workflows (raw ARC label + `container:` directive), see `osdc-workflow-pattern.md`.
11+
12+
## Decide: dial-up vs. 100% opt-in
13+
14+
| Pattern | When to use | Example |
15+
|---|---|---|
16+
| **Dial-up** (preferred) | Existing workflow with broad coverage; you want to ramp OSDC adoption via labels like `pull.yml`/`trunk.yml`. | `test-b200.yml` (PR #181544) |
17+
| **100% opt-in** | Workflow is meant to *always* run on OSDC (e.g. testing the single B200 we own on EKS). | `operator_microbenchmark.yml`, `attention_op_microbenchmark.yml` (B200 jobs) |
18+
19+
The two pieces — `runner_prefix` and `use-arc`**must move together**. Hardcoding one but driving the other off the determinator is a bug. See `feedback-osdc-migration-dial-up.md`.
20+
21+
## Migration steps (dial-up pattern)
22+
23+
Worked reference: PR #181544 / commit `f156b7ddfd1` ("Migrate smoke test on B200 to OSDC"). The diff was 10 lines.
24+
25+
### 1. Make sure `get-label-type` opts into the ARC experiment
26+
27+
In the `get-label-type` job that calls `_runner-determinator.yml`, add:
28+
29+
```yaml
30+
check_experiments: arc,lf
31+
```
32+
33+
Without this the determinator won't consider the ARC experiment and `use-arc` will always be `false`.
34+
35+
### 2. Plumb four inputs into the build job
36+
37+
On the call to `./.github/workflows/_linux-build.yml`:
38+
39+
```yaml
40+
with:
41+
runner_prefix: "${{ needs.get-label-type.outputs.label-type }}" # likely already present
42+
...existing inputs...
43+
use-arc: ${{ needs.get-label-type.outputs.use-arc == 'true' }}
44+
python-version: "3.10" # match the docker-image-name's python
45+
compiler: gcc11 # match the docker-image-name's compiler
46+
cuda-version: "13.0" # match the docker-image-name's cuda (or "" for CPU)
47+
```
48+
49+
The last three feed `setup-linux` so it can configure the OSDC container env. Read them off the existing `docker-image-name` (e.g. `pytorch-linux-jammy-cuda13.0-cudnn9-py3-gcc11` → py3.10 / gcc11 / cuda13.0).
50+
51+
### 3. Add `get-label-type` to the test job's `needs` and plumb the same inputs
52+
53+
On the call to `./.github/workflows/_linux-test.yml`:
54+
55+
```yaml
56+
needs:
57+
- <existing-build-job>
58+
- get-label-type # add this
59+
with:
60+
...existing inputs...
61+
use-arc: ${{ needs.get-label-type.outputs.use-arc == 'true' }}
62+
python-version: "3.10"
63+
compiler: gcc11
64+
cuda-version: "13.0"
65+
```
66+
67+
The test job needs a direct `needs: get-label-type` dependency to read its outputs (it can't rely on transitive `needs` through the build job).
68+
69+
### 4. Drop the OSDC-incompatible inputs
70+
71+
If the EC2 path used inputs that don't apply on OSDC (e.g. `aws-role-to-assume:` for ECR pulls), remove them. OSDC has its own AWS role wired into `setup-linux` (`arn:aws:iam::308535385114:role/arc`) and pulls images from `ghcr.io/pytorch`, not ECR.
72+
73+
In #181544 this meant deleting:
74+
```yaml
75+
aws-role-to-assume: arn:aws:iam::308535385114:role/gha_workflow_s3_and_ecr_read_only
76+
```
77+
78+
### 5. Verify
79+
80+
- `gh workflow run` or push a PR with the relevant `ciflow/*` tag.
81+
- Look for the `build-osdc` / `test-osdc` jobs running (they're separate jobs in `_linux-build.yml` / `_linux-test.yml`, gated on `inputs.use-arc`). When `use-arc` is `false`, the original `build` / `test` jobs run instead.
82+
83+
## Migration steps (100% opt-in pattern)
84+
85+
For a job that should always run on OSDC, skip the determinator-driven plumbing and hardcode:
86+
87+
```yaml
88+
with:
89+
runner_prefix: "mt-"
90+
use-arc: true
91+
python-version: "3.10"
92+
compiler: gcc11
93+
cuda-version: "13.0"
94+
```
95+
96+
Both jobs (build *and* test) need the four inputs. The build job's `runner:` stays as the EC2 label (e.g. `linux.r7i.4xlarge`); `_linux-build.yml`'s `build-osdc` job translates that to the right ARC runner via `map_ec2_to_arc.py`.
97+
98+
Reference: `operator_microbenchmark.yml` jobs `opmicrobenchmark-build-b200` / `opmicrobenchmark-test-b200`.
99+
100+
## What the inputs do (mental model)
101+
102+
`_linux-build.yml` and `_linux-test.yml` each define **two jobs**:
103+
104+
- `build` / `test` — EC2 path. Runs directly on the runner host VM. Gated on `!inputs.use-arc`.
105+
- `build-osdc` / `test-osdc` — OSDC path. Runs inside a `container:` from `ghcr.io/pytorch/${docker-image-name}`. Gated on `inputs.use-arc`.
106+
107+
The OSDC jobs call `setup-linux` with `use-arc: true` and pass `python-version` / `compiler` / `cuda-version` so the container env matches the build environment. That's why the migration must pass these three inputs — `setup-linux` errors out without them on the OSDC path.
108+
109+
## Common gotchas
110+
111+
- **Forgetting `check_experiments: arc,lf`** → `use-arc` is always `false`, so OSDC never activates and you'll think the migration silently failed.
112+
- **Forgetting `needs: get-label-type` on the test job** → `${{ needs.get-label-type.outputs.use-arc }}` evaluates to empty, dial-up doesn't engage.
113+
- **Mismatched `python-version` / `compiler` / `cuda-version` vs. `docker-image-name`** → container has one toolchain, `setup-linux` configures another, build fails confusingly.
114+
- **Leaving `aws-role-to-assume:` for ECR** → harmless on OSDC (it's only used by EC2 path) but stale and misleading; remove it.
115+
- **Mixing patterns** (dynamic `runner_prefix` + hardcoded `use-arc: true`, or vice versa) — see `feedback-osdc-migration-dial-up.md`. Pick one pattern and apply it consistently.
116+
117+
## Cross-references
118+
119+
- `osdc-workflow-pattern.md` — for *standalone* OSDC workflows (no reusable build/test).
120+
- `feedback-osdc-migration-dial-up.md` — why both inputs must move together.
121+
- `ci-consolidation.md` — what's shared / different between EC2 (`build`/`test`) and OSDC (`build-osdc`/`test-osdc`) jobs in the reusable workflows.
122+
- `arc-cpu-build-test.md` — how the OSDC path actually runs once activated.

0 commit comments

Comments
 (0)