Skip to content

Commit 541a7a3

Browse files
kubafloCopilotPureWeen
authored
Replace PR agent with pr-review skill and decouple comment posting (dotnet#34460)
<!-- Please let the below note in for people that find this PR --> > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## 🏗️ Overview This PR replaces the **monolithic PR agent** with a modular, skill-based **PR review orchestrator**. The legacy 2-file agent is decomposed into 4 dedicated phase skills, each with clear boundaries, inputs, and outputs. > **TL;DR** — One big agent file → four focused skills. Cleaner, composable, easier to maintain. --- ## 🔀 Before → After ``` ┌─────────────────────────────────┐ ┌─────────────────────────────────────────────┐ │ BEFORE (Agent) │ │ AFTER (Skills) │ ├─────────────────────────────────┤ ├─────────────────────────────────────────────┤ │ │ │ │ │ agents/pr.md (310 lines) │ │ skills/pr-review/SKILL.md (orchestrator) │ │ + │ ──► │ ├─ skills/pr-preflight/SKILL.md │ │ agents/pr/post-gate.md (331) │ │ ├─ skills/pr-gate/SKILL.md │ │ agents/pr/SHARED-RULES.md │ │ ├─ try-fix (existing skill) │ │ agents/pr/PLAN-TEMPLATE.md │ │ └─ skills/pr-report/SKILL.md │ │ │ │ │ │ 5 comment scripts │ │ 2-section unified comment │ │ │ │ │ └─────────────────────────────────┘ └─────────────────────────────────────────────┘ ``` --- ## ✨ Key Changes ### 1️⃣ Agent → Skill Architecture | Phase | Skill | Responsibility | |:-----:|:------|:---------------| | **0** | `pr-review` | 🎯 Master orchestrator — sets up branch, invokes phases | | **1** | `pr-preflight` | 📋 Context gathering — reads issue, PR, comments, classifies files | | **2** | `pr-gate` | 🚦 Test verification — tests must FAIL without fix, PASS with fix | | **3** | `try-fix` _(existing)_ | 🔧 Multi-model fix exploration (Sonnet → Opus) | | **4** | `pr-report` | 📊 Final recommendation — APPROVE or REQUEST CHANGES | ### 2️⃣ Decoupled Comment Posting `pr-review` produces **output files only** (`CustomAgentLogsTmp/PRState/`). Comments are posted separately via `ai-summary-comment` — no tight coupling. ### 3️⃣ Unified Comment Model Consolidated from **5 separate comment scripts** → **2 sections** in a single unified comment: | ❌ Removed Script | ✅ Now Part Of | |:-----------------|:--------------| | `post-try-fix-comment.ps1` | PR-REVIEW section | | `post-verify-tests-comment.ps1` | PR-REVIEW section | | `post-write-tests-comment.ps1` | PR-REVIEW section | `post-pr-finalize-comment.ps1` now injects into the unified comment by default (use `-Standalone` for legacy behavior). ### 4️⃣ Simplified CI & Orchestration **`Review-PR.ps1`** went from complex 3-phase script → clean 4-step orchestrator: ``` Step 1 → pr-review skill (all 4 phases) Step 2 → pr-finalize skill (verify title/description) Step 3 → ai-summary-comment (post unified comment) Step 4 → Update-AgentLabels (apply labels) ``` **`ci-copilot.yml`** — removed explicit `-RunFinalize -PostSummaryComment` flags (all phases run automatically). --- ## 📁 Files Changed <details> <summary><b>✅ Added (4 files)</b></summary> | File | Purpose | |:-----|:--------| | `.github/skills/pr-review/SKILL.md` | Master orchestrator (Phase 0–4) | | `.github/skills/pr-preflight/SKILL.md` | Phase 1: Context gathering | | `.github/skills/pr-gate/SKILL.md` | Phase 2: Test verification gate | | `.github/skills/pr-report/SKILL.md` | Phase 4: Final recommendation | </details> <details> <summary><b>❌ Deleted (7 files)</b></summary> | File | Reason | |:-----|:-------| | `.github/agents/pr.md` | Replaced by `pr-review` skill | | `.github/agents/pr/PLAN-TEMPLATE.md` | Skills use direct output files | | `.github/agents/pr/SHARED-RULES.md` | Rules distributed to phase skills | | `.github/agents/pr/post-gate.md` | Replaced by `pr-gate` + `pr-report` | | `post-try-fix-comment.ps1` | Merged into unified PR-REVIEW comment | | `post-verify-tests-comment.ps1` | Merged into unified PR-REVIEW comment | | `post-write-tests-comment.ps1` | Not part of main review workflow | </details> <details> <summary><b>🔧 Modified (14 files)</b></summary> | File | Change | |:-----|:-------| | `.github/scripts/Review-PR.ps1` | Major refactor → 4-step skill orchestration | | `.github/skills/ai-summary-comment/SKILL.md` | Simplified to 2-section model | | `.github/skills/ai-summary-comment/.../post-ai-summary-comment.ps1` | Updated for unified comment | | `.github/skills/ai-summary-comment/.../post-pr-finalize-comment.ps1` | Injects into unified comment | | `.github/copilot-instructions.md` | Replaced agent entry with skill entries | | `.github/README-AI.md` | Updated architecture docs | | `.github/docs/agent-labels.md` | Terminology: "Gate" → "Validate" | | `.github/scripts/shared/Update-AgentLabels.ps1` | Updated for phase structure | | `.github/scripts/shared/Start-Emulator.ps1` | Minor utility updates | | `.github/skills/pr-finalize/SKILL.md` | Minor context update | | `.github/skills/verify-tests-fail-without-fix/SKILL.md` | Minor context update | | `.github/skills/ai-summary-comment/NO-EXTERNAL-REFERENCES-RULE.md` | Updated for 2-section model | | `.github/instructions/sandbox.instructions.md` | Updated references | | `eng/pipelines/ci-copilot.yml` | Simplified invocation | </details> --- ## 📊 Impact ``` 25 files changed 843 insertions(+) 3,271 deletions(-) ``` Net reduction of **~2,400 lines** — less code, better structure. 🎉 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
1 parent e2f570f commit 541a7a3

30 files changed

+1293
-5150
lines changed

.github/README-AI.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ This folder contains instructions and configurations for AI coding assistants wo
44

55
## Available Agents
66

7-
### PR Agent
8-
The PR agent is a unified 4-phase workflow for investigating issues and reviewing/working on PRs. It handles everything from context gathering through test verification, fix exploration, and creating PRs or review reports.
7+
### PR Review Agent
8+
The pr-review skill is a 4-phase orchestrator for investigating issues and reviewing/working on PRs. It invokes dedicated phase skills (pr-preflight, pr-gate, try-fix, pr-report) for context gathering, test verification, fix exploration, and review reports.
99

1010
### Sandbox Agent
1111
The sandbox agent is your general-purpose tool for working with the .NET MAUI Sandbox app. Use it for manual testing, PR validation, issue reproduction, and experimentation with MAUI features.
@@ -84,13 +84,13 @@ copilot
8484
please write UI tests for issue #12345
8585
```
8686

87-
**PR Agent:**
87+
**Try-Fix-Validate Agent:**
8888
```bash
8989
# Start GitHub Copilot CLI with agent support
9090
copilot
9191
92-
# Invoke the pr agent
93-
/agent pr
92+
# Invoke the pr-review skill
93+
/skill pr-review
9494
9595
# Fix an issue or review a PR
9696
please fix issue #12345
@@ -112,7 +112,7 @@ please review https://github.com/dotnet/maui/pull/XXXXX
112112
3. **Choose your agent** from the dropdown:
113113
- `sandbox-agent` for manual testing and experimentation
114114
- `write-tests-agent` for writing tests (invokes appropriate skill)
115-
- `pr` for reviewing and working on existing PRs
115+
- `pr-review skill` for reviewing and working on existing PRs
116116

117117
4. **Enter a task** in the text box:
118118
- For sandbox testing: `Please test PR #32479`
@@ -146,9 +146,9 @@ Automated testing specialist for the .NET MAUI test suite:
146146
4. **Cross-Platform** - Tests on iOS, Android, Windows, and MacCatalyst
147147
5. **Automated Workflow** - Uses `BuildAndRunHostApp.ps1` to handle building, deployment, and logging to `CustomAgentLogsTmp/UITests/`
148148
149-
### PR Agent
149+
### PR Review Agent
150150
151-
Unified 4-phase workflow for issue investigation and PR work:
151+
Unified 4-phase orchestrator for issue investigation and PR work:
152152
153153
1. **Pre-Flight** - Context gathering from issues/PRs
154154
2. **Gate** - Verify tests exist and catch the issue (mandatory checkpoint)
@@ -199,18 +199,18 @@ Agents work with **time budgets as estimates for planning**, not hard deadlines:
199199

200200
## File Structure
201201

202-
### Agent Definitions
203-
- **`agents/pr.md`** - PR workflow phases 1-2 (Pre-Flight, Gate)
204-
- **`agents/pr/post-gate.md`** - PR workflow phases 3-4 (Fix, Report)
202+
### Agent & Skill Definitions
203+
- **`skills/pr-review/SKILL.md`** - PR Review orchestrator (invokes phase docs and try-fix skill)
204+
- **`pr-review/pr-preflight.md`** - Phase 1: Context gathering (phase doc, not a standalone skill)
205+
- **`pr-review/pr-gate.md`** - Phase 2: Test verification (phase doc, not a standalone skill)
206+
- **`pr-review/pr-report.md`** - Phase 4: Final recommendation (phase doc, not a standalone skill)
205207
- **`agents/sandbox-agent.md`** - Sandbox agent for testing and experimentation
206208
- **`agents/write-tests-agent.md`** - Test writing agent (dispatches to skills like write-ui-tests)
207209

208210
### Agent Files
209211

210212
Agent files in the `.github/agents/` directory:
211213

212-
- **`agents/pr.md`** - PR workflow phases 1-2 (Pre-Flight, Gate)
213-
- **`agents/pr/post-gate.md`** - PR workflow phases 3-4 (Fix, Report)
214214
- **`agents/sandbox-agent.md`** - Sandbox app testing and experimentation
215215
- **`agents/write-tests-agent.md`** - Test writing (invokes skills like write-ui-tests)
216216

@@ -250,12 +250,12 @@ Reusable skills in `.github/skills/` that agents can invoke:
250250
- **`verify-tests-fail-without-fix/`** - Verifies UI tests catch bugs (auto-detects mode based on git diff)
251251
- **`write-ui-tests/`** - Creates UI tests for issues following MAUI conventions
252252
- **`write-xaml-tests/`** - Creates XAML unit tests for parsing, XamlC, and source generation issues
253-
- **`azdo-build-investigator/`** - MAUI-specific CI investigation context (works with `ci-analysis` from arcade-skills plugin)
253+
- **`azdo-build-investigator/`** - Investigates CI failures for PRs (build errors, Helix test logs, binlog analysis) via dotnet/arcade-skills plugin
254254

255255
### Recent Improvements (January 2026)
256256

257-
**PR Agent Consolidation:**
258-
1. **Unified PR Agent** - Replaced separate `issue-resolver` and `pr-reviewer` agents with single 4-phase `pr` agent
257+
**Agent Consolidation:**
258+
1. **Unified PR Review Orchestrator** - Replaced separate `issue-resolver` and `pr-reviewer` agents with 4-phase `pr-review` skill that orchestrates `pr-preflight`, `pr-gate`, `try-fix`, and `pr-report` phase skills
259259
2. **try-fix Skill** - New skill for exploring independent fix alternatives with empirical testing
260260
3. **Skills Integration** - Added `verify-tests-fail-without-fix` and `write-ui-tests` skills for reusable test workflows
261261
4. **Agent/Skills Guidelines** - New instruction files for authoring agents and skills
@@ -364,8 +364,8 @@ For issues or questions about the AI agent instructions:
364364
## Metrics
365365

366366
**Agent Files**:
367-
- 4 agent files (pr.md, pr/post-gate.md, sandbox-agent.md, write-tests-agent.md)
368-
- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator)
367+
- 4 agent files (pr-review skill.md, sandbox-agent.md, write-tests-agent.md)
368+
- 5 skills (pr-review, try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator) + 3 phase docs (pr-preflight, pr-gate, pr-report)
369369
- All validated and consistent with consolidated structure
370370

371371
**Automation**:
@@ -383,4 +383,4 @@ For issues or questions about the AI agent instructions:
383383

384384
**Last Updated**: 2026-01-07
385385

386-
**Note**: These instructions are actively being refined based on real-world usage. PR agent consolidation completed January 2026 (unified 4-phase workflow with try-fix skill). Feedback and improvements are welcome!
386+
**Note**: These instructions are actively being refined based on real-world usage. Agent consolidation completed January 2026 (unified 4-phase workflow with try-fix skill). Feedback and improvements are welcome!

0 commit comments

Comments
 (0)