Consolidate PR reviewers into a single expert-reviewer agent#8157
Conversation
Replace the three separate PR reviewer workflows (pr-expert-reviewer, pr-nitpick-reviewer, pr-test-expert-reviewer) with a single combined expert-reviewer agent following the MSBuild pattern: - .github/agents/expert-reviewer.agent.md: Core agent with 21 review dimensions covering correctness, threading, security, API compat, performance, cross-TFM, test quality, style, and more. Usable both locally in VS Code and by GH agentic workflows. - .github/workflows/shared/review-shared.md: Shared permissions, tools, safe-outputs, and delegation logic. - .github/workflows/review.agent.md: /review slash command trigger. - .github/workflows/review-on-open.agent.md: Auto-trigger on PR open. Also updates repo-historian.md consumer table to reference the unified reviewer.
There was a problem hiding this comment.
Pull request overview
This PR consolidates three separate PR reviewer workflows into a single unified “expert-reviewer” agent and introduces two entrypoints for running it (on PR open and via /review). It also updates supporting workflow infrastructure/lockfiles to align with the newer gh-aw compiler output and references the new reviewer in repo-historian documentation.
Changes:
- Add unified reviewer agent definition (
.github/agents/expert-reviewer.agent.md) and shared workflow body (.github/workflows/shared/review-shared.md). - Add two new workflows to trigger the unified review (
review.agent.mdfor/review,review-on-open.agent.mdfor non-draft PR opens). - Remove legacy reviewer workflows and update workflow/lockfile infrastructure to reflect the new compilation/versioning.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/shared/review-shared.md | Shared config + instructions for invoking the unified expert reviewer. |
| .github/workflows/review.agent.md | Slash-command (/review) entrypoint importing shared review logic. |
| .github/workflows/review-on-open.agent.md | Auto-trigger entrypoint for non-draft PR opens importing shared review logic. |
| .github/agents/expert-reviewer.agent.md | New consolidated reviewer agent spec with multi-dimension review workflow. |
| .github/workflows/repo-historian.md | Updates consumer table to reference unified reviewer. |
| .github/workflows/repository-quality-improver.lock.yml | Regenerated lockfile (gh-aw/compiler/runtime updates). |
| .github/workflows/repo-historian.lock.yml | Regenerated lockfile (gh-aw/compiler/runtime updates). |
| .github/workflows/daily-malicious-code-scan.lock.yml | Regenerated lockfile (gh-aw/compiler/runtime updates). |
| .github/workflows/agentics-maintenance.yml | Updates generated maintenance workflow (new operations + version bumps). |
| .github/aw/actions-lock.json | Updates pinned gh-aw action versions. |
| .github/workflows/pr-expert-reviewer.md | Deleted legacy expert reviewer workflow. |
| .github/workflows/pr-nitpick-reviewer.md | Deleted legacy nitpick reviewer workflow. |
| .github/workflows/pr-test-expert-reviewer.md | Deleted legacy test-quality reviewer workflow. |
Copilot's findings
- Files reviewed: 21/26 changed files
- Comments generated: 3
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: PR Nitpick Reviewer
Date: 2026-05-12
Repository: microsoft/testfx
Key Findings
3 minor nitpicks across 2 files — no blocking issues.
| # | File | Finding |
|---|---|---|
| 1 | review.agent.md:18 |
Missing checkout: false — inconsistent with review-on-open.agent.md |
| 2 | expert-reviewer.agent.md:520 |
Non-standard 1b. step numbering in ordered list |
| 3 | expert-reviewer.agent.md:522 |
"batches of 6" overstates last-batch size (21 = 3×6 + 3) |
Positive Highlights
- ✅ Excellent consolidation of three separate workflows into one well-structured agent with 21 clear review dimensions.
- ✅ The sub-agent architecture (parallel per-dimension agents + validation wave) is a solid design pattern.
- ✅ The
review-on-open.agent.mdcorrectly usespull_request_targetwithcheckout: falsefor fork safety. - ✅
review-shared.mdkeeps permissions, tools, and safe-outputs DRY across both entry points. - ✅ The Wave 2 validation step ("prove or disprove" before posting) is a great way to reduce false positives.
Recommendations
All findings are minor (NIT-level). The PR is high quality and well-structured.
Generated by PR Nitpick Reviewer 🔍 Meticulously inspected by Nitpick Reviewer
🔍 Meticulously inspected by PR Nitpick Reviewer 🔍
|
@copilot address review ocmments |
|
@copilot resolve the merge conflicts in this pull request |
…batch size wording Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Resolved conflicts in .github/workflows/issue-arborist.lock.yml by accepting main's enhanced Issue Arborist workflow with REST API fallback and GH_AW_ORIGINAL_GITHUB_API_URL support. Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Merge conflicts resolved in |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
There was a problem hiding this comment.
Copilot's findings
Comments suppressed due to low confidence (1)
.github/agents/expert-reviewer.agent.md:475
- This table states “FluentAssertions style for all”, but the repo uses different assertion libraries depending on the test project (e.g., many use AwesomeAssertions, and some ban it). Update this entry to reflect “follow the test project’s assertion convention / BannedSymbols.txt” to prevent incorrect review recommendations.
| 1 | **Public API Shipping** | Declare in `PublicAPI.Unshipped.txt`. Run `eng/mark-shipped.ps1` to promote. Multi-TFM: `net8.0/`, `net9.0/`, `netstandard2.0/` subfolders. | `eng/mark-shipped.ps1` |
| 2 | **No `init` on Public API** | New public API MUST NOT use `init` accessors. Existing MTP `init` accessors are grandfathered. | `.github/copilot-instructions.md` |
| 3 | **Localization** | `.resx` for user-facing strings. Never edit `.xlf` — build generates them. | `src/*/Strings.resx` |
| 4 | **Test Architecture** | MSTest unit tests use `TestFramework.ForTestingMSTest`. MTP/analyzer tests use MSTest. FluentAssertions style for all. | `test/Utilities/TestFramework.ForTestingMSTest` |
| 5 | **IPC Protocol** | Named pipes, JSON-RPC between test platform and runners. Wire format backward-compatible. | `src/Platform/` |
| 6 | **Analyzer IDs** | `MSTEST0001`+ for MSTest analyzers. Unique across codebase. | `src/Analyzers/` |
- Files reviewed: 22/25 changed files
- Comments generated: 4
|
@copilot address review comments |
- Add explicit noop safe-outputs with report-as-issue: false to review workflows - Update CodeQL action metadata to v4.35.4 in lock file headers - Change assertion guidance to follow test project's BannedSymbols.txt policy instead of hardcoding FluentAssertions Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
All review comments addressed:
|
Summary
Replaces the three separate PR reviewer workflows (
pr-expert-reviewer,pr-nitpick-reviewer,pr-test-expert-reviewer) with a single combined expert-reviewer agent following the MSBuild pattern.Motivation
.github/agents/expert-reviewer.agent.mdcan be invoked directly in VS Code for local code review — no need to wait for GH agentic workflows.What changed
New files
.github/agents/expert-reviewer.agent.md— Core agent with 21 review dimensions covering correctness, threading, security, API compat, performance, cross-TFM, test quality, style, and more. Multi-wave workflow (Load History → Find → Validate → Post → Summary)..github/workflows/shared/review-shared.md— Shared permissions, tools, safe-outputs, and delegation logic..github/workflows/review.agent.md—/reviewslash command trigger..github/workflows/review-on-open.agent.md— Auto-trigger on non-draft PR open.Deleted files
.github/workflows/pr-expert-reviewer.md+.lock.yml.github/workflows/pr-nitpick-reviewer.md+.lock.yml.github/workflows/pr-test-expert-reviewer.md+.lock.ymlModified files
.github/workflows/repo-historian.md— Updated consumer table to reference the unified reviewer..github/workflows/daily-malicious-code-scan.lock.yml— Updated CodeQL action to v4.35.4 to align withactions-lock.json..github/workflows/issue-arborist.lock.yml— Merged main's enhanced workflow with REST API fallback.Review dimensions (21)
Review feedback addressed
noopsafe-outputs withreport-as-issue: falseto both review workflows to prevent clean/no-op runs from being escalated as issuesBannedSymbols.txtpolicies (some test projects use AwesomeAssertions, others use MSTest Assert)checkout: falseconsistency, fixed batch size wording, and converted historical context note to blockquote callout