You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Allow fork PRs to auto-trigger evaluate-pr-tests workflow (dotnet#34655)
## Summary
Enables the copilot-evaluate-tests gh-aw workflow to run on fork PRs by
adding `forks: ["*"]` to the `pull_request` trigger and removing the
fork guard from `Checkout-GhAwPr.ps1`.
## Changes
1. **copilot-evaluate-tests.md**: Added `forks: ["*"]` to opt out of
gh-aw auto-injected fork activation guard. Scoped `Checkout-GhAwPr.ps1`
step to `workflow_dispatch` only (redundant for other triggers since
platform handles checkout).
2. **copilot-evaluate-tests.lock.yml**: Recompiled via `gh aw compile` —
fork guard removed from activation `if:` conditions.
3. **Checkout-GhAwPr.ps1**: Removed the `isCrossRepository` fork guard.
Updated header docs and restore comments to accurately describe behavior
for all trigger×fork combinations (including corrected step ordering).
4. **gh-aw-workflows.instructions.md**: Updated all stale references to
the removed fork guard. Documented `forks: ["*"]` opt-in, clarified
residual risk model for fork PRs, and updated troubleshooting table.
## Security Model
Fork PRs are safe because:
- Agent runs in **sandboxed container** with all credentials scrubbed
- Output limited to **1 comment** via `safe-outputs: add-comment: max:
1`
- Agent **prompt comes from base branch** (`runtime-import`) — forks
cannot alter instructions
- Pre-flight check catches missing `SKILL.md` if fork isn't rebased on
`main`
- No workspace code is executed with `GITHUB_TOKEN` (checkout without
execution)
## Testing
- ✅ `workflow_dispatch` tested against fork PR dotnet#34621
- ✅ Lock.yml statically verified — fork guard removed from `if:`
conditions
- ⏳ `pull_request` trigger on fork PRs can only be verified post-merge
(GitHub Actions reads lock.yml from default branch)
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy file name to clipboardExpand all lines: .github/instructions/gh-aw-workflows.instructions.md
+19-16Lines changed: 19 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -44,7 +44,9 @@ The prompt is built in the **activation job** via `{{#runtime-import .github/wor
44
44
45
45
### Fork PR Activation Gate
46
46
47
-
`gh aw compile` automatically injects a fork guard into the activation job's `if:` condition: `head.repo.id == repository_id`. This blocks fork PRs on `pull_request` events. This is **platform behavior** — do not add it manually.
47
+
By default, `gh aw compile` automatically injects a fork guard into the activation job's `if:` condition: `head.repo.id == repository_id`. This blocks fork PRs on `pull_request` events.
48
+
49
+
To **allow fork PRs**, add `forks: ["*"]` to the `pull_request` trigger in the `.md` frontmatter. The compiler removes the auto-injected guard from the compiled `if:` conditions. This is safe when the workflow uses the `Checkout-GhAwPr.ps1` pattern (checkout + trusted-infra restore) and the agent is sandboxed.
|`pull_request`| ✅ Yes | Blocked by auto-generated activation gate |
63
+
|`pull_request` (default) | ✅ Yes | Blocked by auto-generated activation gate unless `forks: ["*"]` is set |
64
+
|`pull_request` + `forks: ["*"]`| ✅ Yes | ✅ Works — user steps restore trusted infra before agent runs |
62
65
|`workflow_dispatch`| ❌ Skipped | ✅ Works — user steps handle checkout and restore is final |
63
66
|`issue_comment` (same-repo) | ✅ Yes | ✅ Works — files already on PR branch |
64
-
|`issue_comment` (fork) |N/A | ❌ Blocked by fail-closed fork guard in `Checkout-GhAwPr.ps1`|
67
+
|`issue_comment` (fork) |✅ Yes | ⚠️ Works — `checkout_pr_branch.cjs` re-checks out fork branch after user steps, potentially overwriting restored infra. Acceptable because agent is sandboxed (no credentials, max 1 comment via safe-outputs). Pre-flight check catches missing `SKILL.md` if fork isn't rebased.|
65
68
66
69
### The `issue_comment` + Fork Problem
67
70
68
-
For `/slash-command` triggers on fork PRs, `checkout_pr_branch.cjs` runs AFTER all user steps and re-checks out the fork branch. This overwrites any files restored by user steps (e.g., `.github/skills/`). There is no way to run user steps after platform steps. A fork could include a crafted `SKILL.md` that alters the agent's evaluation behavior.
71
+
For `/slash-command` triggers on fork PRs, `checkout_pr_branch.cjs` runs AFTER all user steps and re-checks out the fork branch. This overwrites any files restored by user steps (e.g., `.github/skills/`). A fork could include a crafted `SKILL.md` that alters the agent's evaluation behavior.
69
72
70
-
**Current approach (fail-closed fork guard):**`Checkout-GhAwPr.ps1` checks `isCrossRepository` via `gh pr view` for `issue_comment` triggers. If the PR is from a fork or the API call fails, the script exits with code 1. Fork PRs should use `workflow_dispatch` instead, where `checkout_pr_branch.cjs` is skipped and the user step restore is the final workspace state.
73
+
**Accepted residual risk:**The agent runs in a sandboxed container with all credentials scrubbed. The worst outcome is a manipulated evaluation comment (`safe-outputs: add-comment: max: 1`). The agent has no ability to push code, access secrets, or exfiltrate data. The pre-flight check in the agent prompt catches the case where `SKILL.md`is missing entirely (fork not rebased on `main`).
71
74
72
75
**Upstream issue:**[github/gh-aw#18481](https://github.com/github/gh-aw/issues/18481) — "Using gh-aw in forks of repositories"
73
76
@@ -86,16 +89,16 @@ steps:
86
89
87
90
The script:
88
91
1. Captures the base branch SHA before checkout
89
-
2. **Fork guard** (`issue_comment` only): checks `isCrossRepository` — exits 1 if fork or API failure
90
-
3. Checks out the PR branch via `gh pr checkout`
91
-
4. Deletes `.github/skills/` and `.github/instructions/` (prevents fork-added files)
92
-
5. Restores them from the base branch SHA (best-effort, non-fatal)
92
+
2. Checks out the PR branch via `gh pr checkout`
93
+
3. Deletes `.github/skills/` and `.github/instructions/` (prevents fork-added files)
94
+
4. Restores them from the base branch SHA (best-effort, non-fatal)
93
95
94
96
**Behavior by trigger:**
95
-
- **`workflow_dispatch`**: Fork guard skipped. Platform checkout is skipped, so the restore IS the final workspace state (trusted files from base branch)
96
-
- **`issue_comment`** (same-repo): Fork guard passes. Platform re-checks out PR branch — files already match, effectively a no-op
97
-
- **`issue_comment`** (fork): Fork guard rejects — exits 1 with actionable notice to use `workflow_dispatch`
98
-
- **`pull_request`** (same-repo): Fork guard skipped. Files already exist, restore is a no-op
97
+
- **`workflow_dispatch`**: Platform checkout is skipped, so the restore IS the final workspace state (trusted files from base branch)
98
+
- **`pull_request`** (same-repo): User step restores trusted infra. `checkout_pr_branch.cjs` runs after and re-checks out PR branch — for same-repo PRs, skill files typically match main unless the PR modified them.
99
+
- **`pull_request`** (fork with `forks: ["*"]`): Same as above, but fork's skill files may differ. Same residual risk as `issue_comment` fork case — agent is sandboxed, pre-flight catches missing `SKILL.md`.
100
+
- **`issue_comment`** (same-repo): Platform re-checks out PR branch — files already match, effectively a no-op
101
+
- **`issue_comment`** (fork): Platform re-checks out fork branch after us, overwriting restored files. Agent is sandboxed; pre-flight in the prompt catches missing `SKILL.md`
99
102
100
103
### Anti-Patterns
101
104
@@ -197,7 +200,7 @@ Manual triggers (`workflow_dispatch`, `issue_comment`) should bypass the gate. N
197
200
198
201
| What | Behavior | Workaround |
199
202
|------|----------|------------|
200
-
| User steps always before platform steps | Cannot run user code after `checkout_pr_branch.cjs` | Use `workflow_dispatch` for fork PRs; see [gh-aw#18481](https://github.com/github/gh-aw/issues/18481) |
203
+
| User steps always before platform steps | Cannot run user code after `checkout_pr_branch.cjs` | For `issue_comment` fork PRs, accept sandboxed residual risk; see [gh-aw#18481](https://github.com/github/gh-aw/issues/18481) |
201
204
| `--allow-all-tools` in lock.yml | Emitted by `gh aw compile` | Cannot override from `.md` source |
202
205
| MCP integrity filtering | Fork PRs blocked as "unapproved" | Use `steps:` checkout instead of MCP |
203
206
| `gh` CLI inside agent | Credentials scrubbed | Use `steps:` for API calls, or MCP tools |
@@ -215,8 +218,8 @@ Manual triggers (`workflow_dispatch`, `issue_comment`) should bypass the gate. N
215
218
| Symptom | Cause | Fix |
216
219
|---------|-------|-----|
217
220
| Agent evaluates wrong PR | `workflow_dispatch` checks out workflow branch | Add `gh pr checkout` in `steps:` |
218
-
| Agent can't find SKILL.md | Fork PR branch doesn't have `.github/skills/` | Agent posts "rebase or use `workflow_dispatch`" message; or rebase fork on `main` |
219
-
| Fork PR rejected on `/evaluate-tests` | Fail-closed fork guard in `Checkout-GhAwPr.ps1` | Use `workflow_dispatch` with `pr_number` input instead |
221
+
| Agent can't find SKILL.md | Fork PR branch doesn't include `.github/skills/` | Rebase fork on `main`, or use `workflow_dispatch` with `pr_number` input |
222
+
| Fork PR skipped on `pull_request` | `forks: ["*"]` not in workflow frontmatter | Add `forks: ["*"]` under `pull_request:` in the `.md` source and recompile |
220
223
| `gh` commands fail in agent | Credentials scrubbed inside container | Move to `steps:` section |
221
224
| Lock file out of date | Forgot to recompile | Run `gh aw compile` |
Write-Host"❌ Could not verify PR origin — failing closed"
54
-
exit1
55
-
}
56
-
if ($isFork-eq'true') {
57
-
Write-Host"::notice::Fork PR detected — /evaluate-tests via issue_comment is not supported for fork PRs. Use workflow_dispatch with pr_number=$PrNumber instead."
58
-
exit1
59
-
}
60
-
}
61
-
62
45
# ── Save base branch SHA ─────────────────────────────────────────────────────
63
46
# Must be captured BEFORE checkout replaces HEAD.
64
47
# Exported for potential use by downstream platform steps (e.g., checkout_pr_branch.cjs)
@@ -82,11 +65,9 @@ Write-Host "✅ Checked out PR #$PrNumber"
82
65
git log --oneline -1
83
66
84
67
# ── Restore agent infrastructure from base branch ────────────────────────────
85
-
# Best-effort restore of skill/instruction files from the base branch.
86
-
# - workflow_dispatch: platform checkout is skipped, so this IS the final state
87
-
# - issue_comment (same-repo): platform's checkout_pr_branch.cjs runs after and
88
-
# overwrites, but files already match (same repo). Fork PRs are blocked above.
89
-
# - pull_request (same-repo): files already exist, this is a no-op
68
+
# This script only runs for workflow_dispatch (other triggers use the platform's
69
+
# checkout_pr_branch.cjs instead). For workflow_dispatch the platform checkout is
70
+
# skipped, so this restore IS the final workspace state.
90
71
# rm -rf first to prevent fork-added files from surviving the restore.
91
72
92
73
if (Test-Path'.github/skills/') { Remove-Item-Recurse -Force '.github/skills/' }
0 commit comments