Skip to content

Commit a6fa78f

Browse files
can-gaa-houcan-gaa-hou
authored
[CRCR] Add script and workflow to auto-update crcr Terrafile tag on release (#8464)
## Summary Auto-updates `pytorch/ci-infra`'s `crcr/Terrafile` when a test-infra release changes CRCR lambda code. ## Changes - New workflow `update-crcr-terrafile.yml`: on `release: published`, detects if the release touched `aws/lambda/cross_repo_ci_relay/**` and, if so, opens a PR in ci-infra bumping the Terrafile `tag`. - New script `update_crcr_terrafile.py`: rewrites only the `crcr:` `tag:` line in the Terrafile. ## Required GitHub configuration - **Secret `GH_PYTORCHBOT_TOKEN`**: must have `contents: write` and `pull-requests: write` on `pytorch/ci-infra` - **Environment `trigger-nightly`**: used by the new job to access the secret above.
1 parent 0e82c9f commit a6fa78f

2 files changed

Lines changed: 163 additions & 0 deletions

File tree

‎.github/scripts/update_crcr_terrafile.py‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env python3
2+
"""Update the `crcr:` tag in a ci-infra `crcr/Terrafile`.
3+
4+
Rewrites only the `tag:` line inside the `crcr:` block, leaving the
5+
`terraform-aws-vpc` block (and anything else) untouched.
6+
7+
Usage:
8+
update_crcr_terrafile.py <Terrafile> <new_tag>
9+
"""
10+
11+
import re
12+
import sys
13+
14+
15+
def update(path: str, new_tag: str) -> bool:
16+
with open(path) as f:
17+
text = f.read()
18+
19+
# Match the crcr: block and its quoted tag: line. The block is anchored at
20+
# the start of a line ("crcr:") and consists of the following indented
21+
# lines. Only the tag: line inside it is rewritten.
22+
pattern = re.compile(
23+
r'(?P<block>^crcr:\n(?:[ \t].*\n)*?)(?P<indent>[ \t]+)tag:[ \t]*"[^"]*"\n',
24+
re.MULTILINE,
25+
)
26+
27+
def repl(m: re.Match) -> str:
28+
return f'{m.group("block")}{m.group("indent")}tag: "{new_tag}"\n'
29+
30+
new_text, n = pattern.subn(repl, text, count=1)
31+
if n == 0:
32+
print(
33+
f"ERROR: could not find a `crcr:` block with a quoted `tag:` line in {path}",
34+
file=sys.stderr,
35+
)
36+
return False
37+
38+
with open(path, "w") as f:
39+
f.write(new_text)
40+
return True
41+
42+
43+
def main() -> int:
44+
if len(sys.argv) != 3:
45+
print("usage: update_crcr_terrafile.py <Terrafile> <new_tag>", file=sys.stderr)
46+
return 2
47+
return 0 if update(sys.argv[1], sys.argv[2]) else 1
48+
49+
50+
if __name__ == "__main__":
51+
sys.exit(main())

‎.github/workflows/update-crcr-terrafile.yml‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
name: Update ci-infra crcr Terrafile
2+
3+
# When a test-infra lambda release is published, check whether it contains
4+
# changes to the CRCR lambda code (aws/lambda/cross_repo_ci_relay/**). If so,
5+
# open a PR in pytorch/ci-infra that bumps the `tag` in crcr/Terrafile so the
6+
# deploy pulls the newly built cross-repo-ci-webhook.zip / cross-repo-ci-callback.zip.
7+
#
8+
# Not every release needs a bump: only releases that actually changed the CRCR
9+
# lambda code vs the previous lambda release trigger a PR.
10+
11+
on:
12+
release:
13+
types: [published]
14+
workflow_dispatch:
15+
inputs:
16+
tag:
17+
description: "test-infra release tag to promote to ci-infra crcr/Terrafile"
18+
required: true
19+
type: string
20+
21+
permissions:
22+
contents: read
23+
24+
jobs:
25+
maybe-update-crcr-terrafile:
26+
runs-on: ubuntu-latest
27+
environment: trigger-nightly
28+
steps:
29+
- name: Resolve release tag
30+
id: tag
31+
env:
32+
EVENT_TAG: ${{ github.event.release.tag_name }}
33+
INPUT_TAG: ${{ inputs.tag }}
34+
run: |
35+
TAG="${INPUT_TAG:-$EVENT_TAG}"
36+
# Only lambda release tags (vYYYYMMDD-HHMMSS) are eligible. Skips
37+
# -custom tags (test builds) and unrelated releases (e.g. blast-cli-*).
38+
if [[ ! "$TAG" =~ ^v[0-9]{8}-[0-9]{6}$ ]]; then
39+
echo "Skipping non-lambda release tag: $TAG"
40+
echo "skip=true" >> "$GITHUB_OUTPUT"
41+
exit 0
42+
fi
43+
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
44+
echo "skip=false" >> "$GITHUB_OUTPUT"
45+
46+
- name: Checkout test-infra at release tag
47+
if: steps.tag.outputs.skip == 'false'
48+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
49+
with:
50+
ref: ${{ steps.tag.outputs.tag }}
51+
fetch-depth: 0
52+
path: test-infra
53+
54+
# Detect whether this release's commit actually touched CRCR lambda
55+
# files. Only releases whose commit changed aws/lambda/cross_repo_ci_relay/**
56+
# need a Terrafile bump; everything else is skipped.
57+
- name: Detect CRCR lambda changes
58+
id: crcr
59+
if: steps.tag.outputs.skip == 'false'
60+
working-directory: test-infra
61+
env:
62+
NEW_TAG: ${{ steps.tag.outputs.tag }}
63+
run: |
64+
set -euo pipefail
65+
if ! git rev-parse --verify -q "${NEW_TAG}~1" >/dev/null 2>&1; then
66+
echo "Release tag has no parent commit; skipping"
67+
echo "changed=false" >> "$GITHUB_OUTPUT"
68+
exit 0
69+
fi
70+
if git diff --name-only "${NEW_TAG}~1" "${NEW_TAG}" -- aws/lambda/cross_repo_ci_relay/ | grep -q .; then
71+
echo "CRCR lambda code changed in this release; ci-infra Terrafile needs a bump"
72+
echo "changed=true" >> "$GITHUB_OUTPUT"
73+
else
74+
echo "No CRCR lambda changes in this release; no Terrafile update needed"
75+
echo "changed=false" >> "$GITHUB_OUTPUT"
76+
fi
77+
78+
- name: Checkout ci-infra
79+
if: steps.crcr.outputs.changed == 'true'
80+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
81+
with:
82+
repository: pytorch/ci-infra
83+
token: ${{ secrets.GH_PYTORCHBOT_TOKEN }}
84+
ref: main
85+
path: ci-infra
86+
fetch-depth: 0
87+
88+
- name: Update crcr Terrafile tag
89+
if: steps.crcr.outputs.changed == 'true'
90+
working-directory: ci-infra
91+
run: |
92+
set -euo pipefail
93+
python3 ../test-infra/.github/scripts/update_crcr_terrafile.py crcr/Terrafile "${{ steps.tag.outputs.tag }}"
94+
echo "--- crcr/Terrafile diff ---"
95+
git diff -- crcr/Terrafile
96+
97+
- name: Create Pull Request
98+
if: steps.crcr.outputs.changed == 'true'
99+
uses: peter-evans/create-pull-request@22a9089034f40e5a961c8808d113e2c98fb63676 # v7
100+
with:
101+
token: ${{ secrets.GH_PYTORCHBOT_TOKEN }}
102+
path: ci-infra
103+
base: main
104+
branch: create-pull-request/update-crcr-terrafile-${{ steps.tag.outputs.tag }}
105+
add-paths: crcr/Terrafile
106+
commit-message: "Update crcr Terrafile to release ${{ steps.tag.outputs.tag }}"
107+
title: "[CRCR] Update crcr Terrafile to release ${{ steps.tag.outputs.tag }}"
108+
reviewers: fffrog, can-gaa-hou, atalman
109+
body: |
110+
This PR is auto-generated by the `update-crcr-terrafile` workflow in [pytorch/test-infra](https://github.com/pytorch/test-infra).
111+
112+
test-infra release `${{ steps.tag.outputs.tag }}` contains changes to the CRCR (`aws/lambda/cross_repo_ci_relay`) Lambda code, so `crcr/Terrafile` is updated to reference the new release tag and pick up the freshly built `cross-repo-ci-webhook.zip` / `cross-repo-ci-callback.zip` assets.

0 commit comments

Comments
 (0)