Skip to content

Commit 5fc13df

Browse files
committed
Deploy greenlight scan as AWS Lambda; move to py3.13
- Add lambda_handler.py: loads secrets from Secrets Manager, mints a least-privilege App token, and runs one review scan via cli.main - Add greenlight-lambda-release.yml and `just package` to build/publish the dist/greenlight-scan.zip (linux x86_64 / cp313 wheels) - Downgrade Python 3.14 -> 3.13 across mise, pyproject, workflows, uv.lock, and docs - Fix guards.py except tuple syntax (parenthesized) now that py3.14's PEP 758 bare-tuple form is no longer valid on 3.13 - Repoint greenlight-review.yml to the manual / recheck entry point and document the Lambda deployment flow (CLAUDE.md, README, CHEATSHEET) The periodic pytorch/pytorch PR scan now runs as the greenlight-scan Lambda (pytorch-gha-infra-2, us-east-1, EventBridge rate(5 minutes)) instead of a scheduled GHA workflow. The handler sets PYTORCH_GREENLIGHT_MAX_RUNTIME_SECONDS=0 to disable the single-instance lock and both hang guards (the watchdog's os._exit is wrong under Lambda), relying on reserved_concurrent_executions=1 and the Lambda function timeout instead. The 3.13 move dodges a fleet-wide AWS provider 6.x bump that rejects 3.14; scan logic is unchanged. Signed-off-by: Jean Schmidt <contato@jschmidt.me>
1 parent e89b6ee commit 5fc13df

17 files changed

Lines changed: 596 additions & 232 deletions
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: greenlight-lambda-release
2+
3+
on:
4+
workflow_dispatch: {}
5+
6+
permissions:
7+
contents: write
8+
9+
defaults:
10+
run:
11+
working-directory: greenlight
12+
13+
jobs:
14+
release:
15+
name: Build and publish greenlight lambda release
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
19+
- uses: jdx/mise-action@dad1bfd3df957f44999b559dd69dc1671cb4e9ea # v4.2.1
20+
with:
21+
working_directory: greenlight
22+
- name: Sync dependencies
23+
run: just setup
24+
- name: Package lambda zip
25+
run: just package
26+
- name: Smoke-test the built zip imports on Python 3.13
27+
# -S drops site-packages so imports resolve only from the extracted zip, not the
28+
# project .venv that mise puts on sys.path; the test exercises the artifact itself.
29+
run: |
30+
set -euo pipefail
31+
smoke_dir="$(mktemp -d)"
32+
unzip -q dist/greenlight-scan.zip -d "${smoke_dir}"
33+
PYTHONPATH="${smoke_dir}" mise exec -- python -S -c \
34+
"import greenlight.lambda_handler, greenlight.cli, boto3, github, clickhouse_connect, yaml"
35+
- name: Compute release tag
36+
id: tag
37+
run: |
38+
echo "tag=greenlight-lambda-v$(date -u +'%Y%m%d-%H%M%S')" >> "${GITHUB_OUTPUT}"
39+
- name: Publish GitHub Release
40+
uses: ncipollo/release-action@440c8c1cb0ed28b9f43e4d1d670870f059653174 # v1.16.0
41+
with:
42+
artifacts: greenlight/dist/greenlight-scan.zip
43+
tag: ${{ steps.tag.outputs.tag }}
44+
name: ${{ steps.tag.outputs.tag }}
45+
- name: Report release tag
46+
env:
47+
TAG: ${{ steps.tag.outputs.tag }}
48+
run: |
49+
echo "Published greenlight lambda release: ${TAG}"
50+
{
51+
echo "## Greenlight lambda release published"
52+
echo ""
53+
echo "Release tag: \`${TAG}\`"
54+
echo ""
55+
echo "Copy this tag into the pytorch-gha-infra-2 Terrafile to deploy."
56+
} >> "${GITHUB_STEP_SUMMARY}"

.github/workflows/greenlight-pr-review.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -57,10 +57,10 @@ jobs:
5757
with:
5858
fetch-depth: 1
5959

60-
- name: Set up Python 3.14
60+
- name: Set up Python 3.13
6161
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
6262
with:
63-
python-version: "3.14"
63+
python-version: "3.13"
6464

6565
- name: Install uv
6666
uses: ./.github/actions/setup-uv
@@ -357,12 +357,12 @@ jobs:
357357
with:
358358
fetch-depth: 1
359359

360-
- name: Set up Python 3.14
361-
# greenlight pins requires-python >=3.14 with uv python-preference=only-system,
362-
# so uv will not download an interpreter; CI must supply a system 3.14.
360+
- name: Set up Python 3.13
361+
# greenlight pins requires-python >=3.13 with uv python-preference=only-system,
362+
# so uv will not download an interpreter; CI must supply a system 3.13.
363363
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
364364
with:
365-
python-version: "3.14"
365+
python-version: "3.13"
366366

367367
- name: Install uv
368368
uses: ./.github/actions/setup-uv

.github/workflows/greenlight-review.yml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
11
name: Green Light Scan
22

3-
# Manually runs the greenlight scanner (`greenlight review`): lists open pytorch/pytorch PRs
4-
# from trusted authors and dispatches greenlight-pr-review.yml for each new or changed
5-
# PR. Read-only on ClickHouse; dispatches the reviewer workflow via the App token.
3+
# The periodic pytorch/pytorch PR scan runs as the `greenlight-scan` AWS Lambda
4+
# (pytorch-gha-infra-2, us-east-1, EventBridge rate(5 minutes)). This workflow is now the manual /
5+
# `@greenlight recheck` entry point: dispatched by filename from a pytorch/pytorch trigger (or run
6+
# by hand), it runs the same greenlight scanner (`greenlight review`) — lists open pytorch/pytorch
7+
# PRs from trusted authors and dispatches greenlight-pr-review.yml for each new or changed PR.
8+
# Read-only on ClickHouse; dispatches the reviewer workflow via the App token.
69

710
on:
811
workflow_dispatch:

greenlight/CHEATSHEET.md

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ directory. `just` is the front-end for every workflow — run `just` or
55
`just --list` to see all recipes.
66

77
PyTorch Green Light runs one iteration of its `review` phase and exits (cron-like), or
8-
loops as a daemon with `--loop`. It also has a one-shot `verdict` subcommand:
8+
loops as a daemon with `--loop`; in production the scheduled scan runs as the `greenlight-scan`
9+
AWS Lambda (EventBridge `rate(5 minutes)`), with the CLI modes kept for local use. It also has a
10+
one-shot `verdict` subcommand:
911

1012
- `review` — scan the open PRs from a fixed set of trusted authors in `pytorch/pytorch`;
1113
for each, compute its fingerprint (`eval_hash`), read its latest state from
@@ -32,11 +34,11 @@ everything else.
3234

3335
```bash
3436
mise trust # trust greenlight/mise.toml (first use only)
35-
mise install # install python 3.14, uv, just, and the non-Python linters
37+
mise install # install python 3.13, uv, just, and the non-Python linters
3638
just setup # uv sync -> create .venv with the Python deps
3739
```
3840

39-
`mise install` provides python 3.14, uv, just, node, shellcheck, shfmt, taplo,
41+
`mise install` provides python 3.13, uv, just, node, shellcheck, shfmt, taplo,
4042
and markdownlint-cli2. `just setup` then installs the Python tools (ruff, mypy,
4143
pytest, yamllint) into `.venv`.
4244

@@ -111,6 +113,19 @@ and `--lock-path` override the matching env vars, and `review` adds the scan fla
111113
Raise verbosity with `--log-level DEBUG` (or `PYTORCH_GREENLIGHT_LOG_LEVEL=DEBUG`); DEBUG also
112114
logs the resolved `Config`.
113115

116+
## Package and deploy
117+
118+
```bash
119+
just package # build dist/greenlight-scan.zip (linux x86_64 / cp313 wheels) for the Lambda
120+
```
121+
122+
In production the scheduled scan runs as the `greenlight-scan` AWS Lambda
123+
(`pytorch-gha-infra-2`, `us-east-1`, EventBridge `rate(5 minutes)`), not via the CLI or a GHA
124+
workflow. Ship a new build with `just package` -> run the `greenlight-lambda-release.yml`
125+
workflow (publishes a `greenlight-lambda-v<timestamp>` Release with the zip) -> pin that tag in
126+
the `pytorch-gha-infra-2` `runners/common/Terrafile` -> `terraform apply` in
127+
`runners/regions/us-east-1`.
128+
114129
## Simulate a run
115130

116131
The end-to-end flow, per trusted-author PR:

greenlight/CLAUDE.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This file is the canonical project guidance for any coding agent operating in `g
44

55
## What This Is
66

7-
PyTorch Green Light is a Python 3.14 service invoked from the CLI. Its `review` phase
7+
PyTorch Green Light is a Python 3.13 service invoked from the CLI. Its `review` phase
88
runs a single iteration and exits (cron-like), e.g. `just run review`. It can also run
99
as a long-lived daemon with `--loop`, e.g. `just run review --loop`, which repeats the
1010
phase on an interval. The phase runs through the same one-shot and daemon execution
@@ -13,7 +13,7 @@ change to phase logic.
1313

1414
## Tooling
1515

16-
- **mise** provisions every tool (python 3.14, uv, just, shellcheck, shfmt,
16+
- **mise** provisions every tool (python 3.13, uv, just, shellcheck, shfmt,
1717
taplo, markdownlint-cli2, node). `mise install` bootstraps everything.
1818
- **uv** owns the Python venv and dependencies, and runs the Python tools (ruff,
1919
mypy, pytest, yamllint).
@@ -89,6 +89,11 @@ interrupt blocking C calls (e.g. DNS) or off-main-thread work; and a hard watchd
8989
force-exits the process a grace period later, the real backstop for hangs the soft timeout
9090
cannot reach.
9191

92+
In production the scheduled scan instead deploys as the `greenlight-scan` AWS Lambda: it runs
93+
the one-shot path with no single-instance lock and both hang-guard layers off
94+
(`PYTORCH_GREENLIGHT_MAX_RUNTIME_SECONDS=0`; the watchdog's `os._exit` is wrong under Lambda),
95+
relying on `reserved_concurrent_executions=1` and the Lambda function timeout instead.
96+
9297
## Comments
9398

9499
Default is NO comment. Add one only for a genuinely non-obvious, durable WHY. No

greenlight/README.md

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,22 @@
11
# PyTorch Green Light
22

33
A Python service that runs a periodic iteration from the CLI — a one-shot
4-
(cron-like) run by default, or a long-lived daemon with `--loop`.
4+
(cron-like) run by default, or a long-lived daemon with `--loop`. In production the
5+
scheduled scan runs as an AWS Lambda (`greenlight-scan`) on a 5-minute EventBridge
6+
schedule; the CLI one-shot and `--loop` daemon modes remain for local and other use.
57

68
## Requirements
79

810
- **mise** — the only manual prerequisite. See <https://mise.jdx.dev>.
911

10-
Everything else (Python 3.14, uv, just, and the non-Python linters) is provided
12+
Everything else (Python 3.13, uv, just, and the non-Python linters) is provided
1113
by mise; `just setup` then installs the Python tools (ruff, mypy, pytest, yamllint).
1214

1315
## Setup
1416

1517
```bash
1618
mise trust # trust greenlight/mise.toml on first use
17-
mise install # install python 3.14, uv, just, and all tools
19+
mise install # install python 3.13, uv, just, and all tools
1820
just setup # uv sync -> create .venv with deps
1921
```
2022

@@ -147,6 +149,29 @@ iteration in both one-shot and `--loop` mode. In `--loop` mode, SIGTERM/SIGINT a
147149
observed only between iterations, so the per-iteration timeout is what interrupts a
148150
hung run.
149151

152+
## Deployment
153+
154+
In production the scheduled scan runs as an AWS Lambda, `greenlight-scan`, in the
155+
`pytorch-gha-infra-2` account (`us-east-1`), triggered by an EventBridge `rate(5 minutes)`
156+
schedule. The function runs `python3.13` with handler `greenlight.lambda_handler.handler`, a
157+
300 s timeout, and `reserved_concurrent_executions = 1`. It runs the same one-shot
158+
`execute_once` / `review.run` path as `greenlight review` — no scan-logic change — after minting a
159+
least-privilege GitHub App installation token in-process and reading the App PEM and ClickHouse
160+
password from AWS Secrets Manager (`pytorch-greenlight-secrets`) at runtime. The handler sets
161+
`PYTORCH_GREENLIGHT_MAX_RUNTIME_SECONDS=0`, so it runs with no single-instance lock and both
162+
hang-guard layers off (the SIGALRM soft timeout and the `os._exit` hard watchdog, which is wrong
163+
under the Lambda runtime); single-instance and hang-bounding come from
164+
`reserved_concurrent_executions = 1` and the Lambda function timeout instead.
165+
166+
Shipping a new version is a manual four-step flow:
167+
168+
1. `just package` builds `dist/greenlight-scan.zip` (linux x86_64 / cp313 wheels).
169+
2. The `greenlight-lambda-release.yml` workflow (test-infra, manual `workflow_dispatch`) builds
170+
the zip and publishes a `greenlight-lambda-v<timestamp>` GitHub Release with it.
171+
3. An operator pins that release tag in `pytorch-gha-infra-2`'s `runners/common/Terrafile` (the
172+
`greenlight-scan` entry).
173+
4. `terraform apply` in `runners/regions/us-east-1` rolls it out.
174+
150175
## Reviewer checkout sanitizing
151176

152177
The reviewer workflow (`greenlight-pr-review.yml`) checks the PR's `pytorch/pytorch` tree
@@ -266,6 +291,7 @@ src/greenlight/
266291
__init__.py # package exports (Config, __version__)
267292
__main__.py # `python -m greenlight` entry point
268293
cli.py # CLI parsing (review + verdict subcommands), dispatch, exit codes
294+
lambda_handler.py # AWS Lambda entry point: load secrets, mint App token, run one review scan via cli.main
269295
runner.py # run_forever(): resilient daemon loop; execute_once(): one-shot phase run
270296
review.py # scan trusted-author PRs: fingerprint, read state, dispatch reviewer workflow for new/changed; raises on failure
271297
state.py # read a PR's latest recorded state from misc.greenlight_pr_state

greenlight/justfile

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ lint:
5454
_run "taplo check" taplo check
5555
_run "shellcheck" shellcheck --rcfile .shellcheckrc scripts/*.sh ../.claude/hooks/greenlight/*.sh
5656
_run "shfmt" shfmt -d -i 2 -ci scripts
57-
_run "markdownlint" markdownlint-cli2 "**/*.md" "#.venv"
57+
_run "markdownlint" markdownlint-cli2 "**/*.md" "#.venv" "#build" "#dist"
5858
5959
echo ""
6060
if [[ ${#FAILED[@]} -gt 0 ]]; then
@@ -73,7 +73,39 @@ lint-fix:
7373
ruff format .
7474
shfmt -w -i 2 -ci scripts
7575
taplo fmt
76-
markdownlint-cli2 --fix "**/*.md" "#.venv" || true
76+
markdownlint-cli2 --fix "**/*.md" "#.venv" "#build" "#dist" || true
77+
78+
# Build the AWS Lambda deployment zip (dist/greenlight-scan.zip) with linux x86_64 / cp313 wheels
79+
package:
80+
#!/usr/bin/env bash
81+
set -euo pipefail
82+
source scripts/mise-activate.sh
83+
84+
repo="$(pwd)"
85+
build_dir="${repo}/build/lambda"
86+
reqs="${repo}/build/requirements-lambda.txt"
87+
zip_path="${repo}/dist/greenlight-scan.zip"
88+
89+
rm -rf "${build_dir}" "${zip_path}"
90+
mkdir -p "${build_dir}" "${repo}/dist"
91+
92+
uv export --no-dev --no-emit-project --no-hashes --frozen --format requirements.txt -o "${reqs}"
93+
94+
uv run --with pip -- python -m pip install \
95+
-r "${reqs}" \
96+
-t "${build_dir}" \
97+
--platform manylinux2014_x86_64 \
98+
--platform manylinux_2_28_x86_64 \
99+
--only-binary=:all: \
100+
--python-version 3.13 \
101+
--implementation cp
102+
103+
cp -a "${repo}/src/greenlight" "${build_dir}/greenlight"
104+
find "${build_dir}" -name __pycache__ -type d -prune -exec rm -rf {} +
105+
106+
(cd "${build_dir}" && zip -q -r "${zip_path}" .)
107+
108+
echo "Built ${zip_path} ($(du -h "${zip_path}" | cut -f1))"
77109
78110
# Remove caches, build artifacts, and the venv
79111
clean:

greenlight/mise.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[tools]
2-
python = "3.14"
2+
python = "3.13"
33
uv = "latest"
44
"aqua:casey/just" = "latest"
55
"aqua:koalaman/shellcheck" = "latest"

greenlight/pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
name = "greenlight"
33
version = "0.0.0"
44
description = "CLI-invoked periodic iteration service, daemon-capable."
5-
requires-python = ">=3.14,<3.15"
5+
requires-python = ">=3.13,<3.14"
66
dependencies = ["pygithub>=2.6.1", "clickhouse-connect>=0.10", "pyyaml>=6", "boto3>=1.34"]
77

88
[project.scripts]
@@ -23,7 +23,7 @@ python-preference = "only-system"
2323

2424
[tool.ruff]
2525
line-length = 120
26-
target-version = "py314"
26+
target-version = "py313"
2727
src = ["src", "tests"]
2828

2929
[tool.ruff.lint]
@@ -82,7 +82,7 @@ exclude_lines = [
8282
]
8383

8484
[tool.mypy]
85-
python_version = "3.14"
85+
python_version = "3.13"
8686
files = ["src", "tests"]
8787
mypy_path = "src"
8888
explicit_package_bases = true

greenlight/src/greenlight/guards.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ def _on_alarm(*_args: object) -> None:
7777
try:
7878
signal.setitimer(signal.ITIMER_REAL, seconds)
7979
armed = True
80-
except OverflowError, signal.ItimerError:
80+
except (OverflowError, signal.ItimerError):
8181
logger.warning("cannot arm iteration timeout for %s seconds; running without it", seconds)
8282
yield
8383
finally:

0 commit comments

Comments
 (0)