Skip to content

Commit c0b0155

Browse files
subinz1subinz1
authored
[CRCR] Add delivery-id and event-type inputs to callback action (Phase 2) (#8303)
## Summary Phase 2 of the CRCR nightly/periodic self-report implementation ([RFC 98](pytorch/rfcs#98)). Extends the callback action (`.github/actions/cross-repo-ci-relay-callback/action.yml`) to support **self-report mode** for nightly/periodic workflows. Downstream repos that self-trigger via cron (no upstream `repository_dispatch`) can now report results by setting two new inputs: - **`delivery-id`**: The upstream `pytorch/pytorch` commit SHA that was tested — becomes the correlation key on the HUD - **`event-type`**: `"nightly"` or `"periodic"` When both are set, the action constructs the payload from scratch (no `client_payload` needed). Existing PR/push behavior is completely unchanged. **Validation:** - `event-type` must be `"nightly"` or `"periodic"` - `status` must be `"completed"` (single-callback model, no `in_progress` step) **Example downstream usage:** ```yaml - name: Report nightly results to CRCR if: always() uses: pytorch/test-infra/.github/actions/cross-repo-ci-relay-callback@main with: status: completed conclusion: ${{ job.status }} delivery-id: ${{ steps.sha.outputs.sha }} event-type: nightly ``` **Depends on:** Phase 1 (#8302) — callback Lambda handler for nightly/periodic ## Test plan - [ ] CI passes (actionlint) - [ ] End-to-end test with Phase 3 downstream workflow
1 parent fb6a1fc commit c0b0155

3 files changed

Lines changed: 253 additions & 92 deletions

File tree

‎.github/actions/cross-repo-ci-relay-callback/action.yml‎

Lines changed: 24 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,17 @@ description: >
55
Relay server. The job must have `id-token: write` permission so that a
66
GitHub OIDC token can be minted and used to authenticate the callback.
77
8-
This action is meant to run in a workflow triggered by a `repository_dispatch`
9-
event from the relay. It reads the dispatch payload (`github.event.client_payload`)
10-
and the ambient `github` context directly, so workflow authors only need to
11-
supply the relay URL, the status/conclusion, and optional structured test
12-
results.
8+
For PR/push workflows triggered by `repository_dispatch`, the action reads
9+
`github.event.client_payload` automatically. For nightly/periodic workflows
10+
triggered by `schedule` (cron), set `delivery-id` to the upstream commit SHA
11+
and `event-type` to "nightly" or "periodic" — the action will construct the
12+
payload from scratch (no client_payload needed).
1313
1414
inputs:
1515
status:
1616
description: >
1717
Workflow status to report. Must be either "in_progress" or "completed".
18+
For nightly/periodic event types, only "completed" is accepted.
1819
required: true
1920
conclusion:
2021
description: >
@@ -24,6 +25,21 @@ inputs:
2425
when status is "in_progress".
2526
required: false
2627
default: ''
28+
delivery-id:
29+
description: >
30+
Dispatch ID for the callback. For PR/push workflows this is read from
31+
client_payload automatically and should NOT be set. For nightly/periodic
32+
workflows (self-triggered via cron), set this to the upstream commit SHA
33+
that was tested — it becomes the correlation key on the HUD.
34+
required: false
35+
default: ''
36+
event-type:
37+
description: >
38+
Event type override. For PR/push workflows this is read from
39+
client_payload automatically and should NOT be set. For nightly/periodic
40+
workflows, set this to "nightly" or "periodic".
41+
required: false
42+
default: ''
2743
test-results:
2844
description: >
2945
Optional JSON string with test result summary (counts: passed/failed/skipped).
@@ -89,90 +105,6 @@ runs:
89105
RUN_ID: ${{ github.run_id }}
90106
RUN_ATTEMPT: ${{ github.run_attempt }}
91107
MAX_TIME: ${{ inputs.max-time }}
92-
run: |
93-
set -euo pipefail
94-
95-
PAYLOAD=$(python3 - <<'PYEOF'
96-
import json, os, sys
97-
from datetime import datetime, timezone
98-
99-
status = os.environ["STATUS"]
100-
if status not in ("in_progress", "completed"):
101-
sys.exit(f"::error::status must be 'in_progress' or 'completed', got {status!r}")
102-
103-
# Pass the conclusion through as-is for completed runs; GitHub validates
104-
# it when the check run is created. in_progress runs carry no conclusion.
105-
conclusion = os.environ.get("CONCLUSION", "").strip() or None
106-
if status == "in_progress":
107-
conclusion = None
108-
109-
try:
110-
client_payload = json.loads(os.environ["CLIENT_PAYLOAD"])
111-
except json.JSONDecodeError as exc:
112-
sys.exit(f"::error::github.event.client_payload is not valid JSON: {exc}")
113-
114-
current_time = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
115-
116-
# In case check_run_id is not exist (edge case), replace it
117-
# with {run_id}-{run_attempt}, which is also unique for each job run.
118-
check_run_id = os.environ.get("CHECK_RUN_ID", "").strip()
119-
if not check_run_id:
120-
check_run_id = f"{os.environ['RUN_ID']}-{os.environ['RUN_ATTEMPT']}"
121-
122-
# Relay's original dispatch payload (event_type, delivery_id, payload) is
123-
# forwarded verbatim. Downstream-reported fields live in a sibling
124-
# `workflow` dict so the two sources stay clearly separated on the wire.
125-
workflow: dict = {
126-
"schema_version": str(os.environ["SCHEMA_VERSION"]),
127-
"status": status,
128-
"conclusion": conclusion,
129-
"name": os.environ["WORKFLOW_NAME"],
130-
"url": os.environ["WORKFLOW_URL"],
131-
"run_attempt": os.environ["RUN_ATTEMPT"],
132-
"job_name": os.environ["JOB_NAME"],
133-
"check_run_id": check_run_id,
134-
"run_id": str(os.environ["RUN_ID"]),
135-
"started_at": None if status == "completed" else current_time,
136-
"completed_at": None if status == "in_progress" else current_time,
137-
}
138-
139-
test_results = os.environ.get("TEST_RESULTS", "").strip()
140-
if test_results:
141-
try:
142-
workflow["test_results"] = json.loads(test_results)
143-
except json.JSONDecodeError as exc:
144-
sys.exit(f"::error::test-results input is not valid JSON: {exc}")
145-
146-
artifact_url = os.environ.get("ARTIFACT_URL", "").strip()
147-
if artifact_url:
148-
workflow["artifact_url"] = artifact_url
149-
150-
client_payload["workflow"] = workflow
151-
print(json.dumps(client_payload))
152-
PYEOF
153-
)
154-
155-
set +e
156-
HTTP_CODE=$(
157-
curl --silent --show-error --fail-with-body --output /tmp/relay_response.json \
158-
--write-out "%{http_code}" \
159-
-X POST \
160-
--max-time ${MAX_TIME} \
161-
-H "Content-Type: application/json" \
162-
-H "Authorization: Bearer ${OIDC_TOKEN}" \
163-
--data "${PAYLOAD}" \
164-
"${CALLBACK_URL%/}"
165-
)
166-
CURL_EXIT_CODE=$?
167-
set -e
168-
169-
if [[ "${CURL_EXIT_CODE}" -ne 0 ]]; then
170-
echo "::error::Callback server returned HTTP ${HTTP_CODE}."
171-
if [[ -s /tmp/relay_response.json ]]; then
172-
echo "Relay server error response body:"
173-
cat /tmp/relay_response.json
174-
fi
175-
exit "${CURL_EXIT_CODE}"
176-
fi
177-
178-
echo "Relay server response HTTP: ${HTTP_CODE}"
108+
DELIVERY_ID_OVERRIDE: ${{ inputs.delivery-id }}
109+
EVENT_TYPE_OVERRIDE: ${{ inputs.event-type }}
110+
run: python3 "${{ github.action_path }}/report_callback.py"

‎.github/actions/cross-repo-ci-relay-callback/report_callback.py‎

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
#!/usr/bin/env python3
2+
"""
3+
CI-neutral CRCR callback reporter.
4+
5+
Builds the callback payload from environment variables and POSTs it to the
6+
relay server. CI-specific wrappers (GHA composite action, Buildkite step)
7+
set the env vars and mint an OIDC token, then call this script.
8+
9+
Required env vars:
10+
OIDC_TOKEN Bearer token for relay auth
11+
CALLBACK_URL Relay endpoint URL
12+
STATUS "in_progress" or "completed"
13+
WORKFLOW_NAME Human-readable workflow name
14+
WORKFLOW_URL URL to the CI run
15+
RUN_ID Unique run identifier
16+
RUN_ATTEMPT Attempt number (1-based)
17+
JOB_NAME Job identifier for the check run name
18+
SCHEMA_VERSION Payload schema version (currently "1")
19+
20+
Optional env vars:
21+
CONCLUSION Job conclusion (required when STATUS=completed)
22+
CLIENT_PAYLOAD JSON string of the dispatch payload (PR/push mode)
23+
DELIVERY_ID_OVERRIDE Upstream commit SHA (nightly/periodic self-report)
24+
EVENT_TYPE_OVERRIDE "nightly" or "periodic" (self-report)
25+
CHECK_RUN_ID GitHub check run ID (falls back to RUN_ID-RUN_ATTEMPT)
26+
TEST_RESULTS JSON string with test result summary
27+
ARTIFACT_URL URL to downstream artifacts
28+
MAX_TIME curl --max-time (default 10)
29+
"""
30+
31+
import json
32+
import os
33+
import subprocess
34+
import sys
35+
import tempfile
36+
from datetime import datetime, timezone
37+
38+
39+
def build_payload() -> str:
40+
status = os.environ["STATUS"]
41+
delivery_id = os.environ.get("DELIVERY_ID_OVERRIDE", "").strip()
42+
event_type = os.environ.get("EVENT_TYPE_OVERRIDE", "").strip()
43+
44+
if bool(delivery_id) != bool(event_type):
45+
sys.exit(
46+
"Error: Both 'delivery-id' and 'event-type' must be set together "
47+
"for nightly/periodic mode. Got delivery-id="
48+
f"{delivery_id!r}, event-type={event_type!r}"
49+
)
50+
51+
is_self_report = bool(delivery_id and event_type)
52+
53+
if is_self_report:
54+
if event_type not in ("nightly", "periodic"):
55+
sys.exit(
56+
f"Error: event-type must be 'nightly' or 'periodic', got {event_type!r}"
57+
)
58+
if status != "completed":
59+
sys.exit(
60+
f"Error: nightly/periodic callbacks require status 'completed', got {status!r}"
61+
)
62+
63+
if status not in ("in_progress", "completed"):
64+
sys.exit(f"Error: status must be 'in_progress' or 'completed', got {status!r}")
65+
66+
conclusion = os.environ.get("CONCLUSION", "").strip() or None
67+
if status == "in_progress":
68+
conclusion = None
69+
70+
if is_self_report:
71+
client_payload = {
72+
"event_type": event_type,
73+
"delivery_id": delivery_id,
74+
"payload": {
75+
"repository": {"full_name": "pytorch/pytorch"},
76+
"head_sha": delivery_id,
77+
},
78+
}
79+
else:
80+
raw = os.environ.get("CLIENT_PAYLOAD", "null")
81+
try:
82+
client_payload = json.loads(raw)
83+
except json.JSONDecodeError as exc:
84+
sys.exit(f"Error: CLIENT_PAYLOAD is not valid JSON: {exc}")
85+
86+
current_time = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
87+
88+
check_run_id = os.environ.get("CHECK_RUN_ID", "").strip()
89+
if not check_run_id:
90+
check_run_id = f"{os.environ['RUN_ID']}-{os.environ['RUN_ATTEMPT']}"
91+
92+
workflow = {
93+
"schema_version": str(os.environ["SCHEMA_VERSION"]),
94+
"status": status,
95+
"conclusion": conclusion,
96+
"name": os.environ["WORKFLOW_NAME"],
97+
"url": os.environ["WORKFLOW_URL"],
98+
"run_attempt": os.environ["RUN_ATTEMPT"],
99+
"job_name": os.environ["JOB_NAME"],
100+
"check_run_id": check_run_id,
101+
"run_id": str(os.environ["RUN_ID"]),
102+
"started_at": None if status == "completed" else current_time,
103+
"completed_at": None if status == "in_progress" else current_time,
104+
}
105+
106+
test_results = os.environ.get("TEST_RESULTS", "").strip()
107+
if test_results:
108+
try:
109+
workflow["test_results"] = json.loads(test_results)
110+
except json.JSONDecodeError as exc:
111+
sys.exit(f"Error: TEST_RESULTS is not valid JSON: {exc}")
112+
113+
artifact_url = os.environ.get("ARTIFACT_URL", "").strip()
114+
if artifact_url:
115+
workflow["artifact_url"] = artifact_url
116+
117+
client_payload["workflow"] = workflow
118+
return json.dumps(client_payload)
119+
120+
121+
def send_callback(payload: str) -> None:
122+
callback_url = os.environ["CALLBACK_URL"].rstrip("/")
123+
oidc_token = os.environ["OIDC_TOKEN"]
124+
max_time = os.environ.get("MAX_TIME", "10")
125+
126+
with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f:
127+
f.write(payload)
128+
payload_file = f.name
129+
130+
try:
131+
result = subprocess.run(
132+
[
133+
"curl",
134+
"--silent",
135+
"--show-error",
136+
"--fail-with-body",
137+
"--output",
138+
"/tmp/relay_response.json",
139+
"--write-out",
140+
"%{http_code}",
141+
"-X",
142+
"POST",
143+
"--max-time",
144+
max_time,
145+
"-H",
146+
"Content-Type: application/json",
147+
"-H",
148+
f"Authorization: Bearer {oidc_token}",
149+
"--data",
150+
f"@{payload_file}",
151+
callback_url,
152+
],
153+
capture_output=True,
154+
text=True,
155+
)
156+
finally:
157+
os.unlink(payload_file)
158+
159+
http_code = result.stdout.strip()
160+
if result.returncode != 0:
161+
print(f"Error: Callback server returned HTTP {http_code}.", file=sys.stderr)
162+
try:
163+
with open("/tmp/relay_response.json") as f:
164+
body = f.read()
165+
if body:
166+
print(f"Relay server error response body:\n{body}", file=sys.stderr)
167+
except FileNotFoundError:
168+
pass
169+
sys.exit(result.returncode)
170+
171+
print(f"Relay server response HTTP: {http_code}")
172+
173+
174+
if __name__ == "__main__":
175+
payload = build_payload()
176+
send_callback(payload)

‎.github/actions/cross-repo-ci-relay-callback/report_callback_buildkite.sh‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
#!/usr/bin/env bash
2+
# Buildkite wrapper for the CRCR callback reporter.
3+
#
4+
# Maps Buildkite-native env vars to the CI-neutral env vars expected by
5+
# report_callback.py, mints a Buildkite OIDC token, and calls the script.
6+
#
7+
# Usage (in a Buildkite step command):
8+
# DELIVERY_ID_OVERRIDE="<upstream SHA>" \
9+
# EVENT_TYPE_OVERRIDE="nightly" \
10+
# CONCLUSION="success" \
11+
# bash .github/actions/cross-repo-ci-relay-callback/report_callback_buildkite.sh
12+
#
13+
# Required Buildkite env (set automatically by the agent):
14+
# BUILDKITE_BUILD_ID, BUILDKITE_RETRY_COUNT, BUILDKITE_PIPELINE_SLUG,
15+
# BUILDKITE_BUILD_URL, BUILDKITE_LABEL or BUILDKITE_STEP_KEY
16+
#
17+
# Required caller-set env:
18+
# STATUS (default: completed), CONCLUSION, DELIVERY_ID_OVERRIDE, EVENT_TYPE_OVERRIDE
19+
20+
set -euo pipefail
21+
22+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
23+
24+
export SCHEMA_VERSION="${SCHEMA_VERSION:-1}"
25+
export STATUS="${STATUS:-completed}"
26+
export CALLBACK_URL="${CALLBACK_URL:-https://gwuj7w5coh4y4l66urspge6mnm0pwaxq.lambda-url.us-east-1.on.aws/github/callback/}"
27+
export MAX_TIME="${MAX_TIME:-10}"
28+
29+
# Map Buildkite env -> CI-neutral env
30+
export RUN_ID="${BUILDKITE_BUILD_ID}"
31+
export RUN_ATTEMPT=$(( ${BUILDKITE_RETRY_COUNT:-0} + 1 ))
32+
export WORKFLOW_NAME="${BUILDKITE_PIPELINE_SLUG}"
33+
export WORKFLOW_URL="${BUILDKITE_BUILD_URL}"
34+
export JOB_NAME="${BUILDKITE_LABEL:-${BUILDKITE_STEP_KEY:-unknown}}"
35+
export CHECK_RUN_ID="${BUILDKITE_BUILD_ID}-${RUN_ATTEMPT}"
36+
37+
# Map Buildkite exit status -> conclusion (if not already set)
38+
if [[ -z "${CONCLUSION:-}" && -n "${BUILDKITE_COMMAND_EXIT_STATUS:-}" ]]; then
39+
if [[ "${BUILDKITE_COMMAND_EXIT_STATUS}" -eq 0 ]]; then
40+
CONCLUSION="success"
41+
else
42+
CONCLUSION="failure"
43+
fi
44+
fi
45+
export CONCLUSION
46+
47+
# Mint Buildkite OIDC token (same audience as GitHub OIDC)
48+
export OIDC_TOKEN
49+
OIDC_TOKEN=$(buildkite-agent oidc request-token \
50+
--audience "pytorch-cross-repo-ci-relay" \
51+
--lifetime 300)
52+
53+
python3 "${SCRIPT_DIR}/report_callback.py"

0 commit comments

Comments
 (0)