Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/ISSUE_TEMPLATE/bug-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,13 @@ body:
label: Version with bug
description: In what version do you see this issue? Run `dotnet workload list` to find your version.
options:
- 11.0.0-preview.6
- 11.0.0-preview.5
- 11.0.0-preview.4
- 11.0.0-preview.3
- 11.0.0-preview.2
- 11.0.0-preview.1
- 10.0.90
- 10.0.80
- 10.0.71
- 10.0.70
Expand Down Expand Up @@ -176,11 +178,13 @@ body:
- 10.0.70
- 10.0.71
- 10.0.80
- 10.0.90
- 11.0.0-preview.1
- 11.0.0-preview.2
- 11.0.0-preview.3
- 11.0.0-preview.4
- 11.0.0-preview.5
- 11.0.0-preview.6
validations:
required: true
- type: dropdown
Expand Down
3 changes: 2 additions & 1 deletion .github/agents/release-readiness-agent.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,8 @@ Both lanes may emit `UNKNOWN` rows when a tool isn't available in the running en
| `UNKNOWN` row | MCP tool | Patch rule |
|---|---|---|
| `BAR default-channel mapping (<branch> → .NET <band> SDK)` | `maestro_default_channels` with `repository: https://github.com/dotnet/maui` | Mapping present + enabled → `READY`. Missing/disabled → `BLOCKED` + surface the `darc add-default-channel` command from the script's `Next action`. |
| `BAR build for <branch> HEAD (<short-sha>)` | `maestro_builds` with `commit: <full-sha>` and `repository: https://github.com/dotnet/maui` | ≥1 build returned → `READY` and cite buildNumber/id. Empty → `WATCH` (transient, CI still running). |
| `BAR build for <branch> HEAD (<short-sha>)` | `maestro_builds` with `commit: <full-sha>` and `repository: https://github.com/dotnet/maui` | Filter returned builds to the requested SR branch **before** patching: accept only builds whose branch metadata is `<branch>` (or `refs/heads/<branch>`). A same-SHA build from `main` or another branch does **not** count. Matching SR build → `READY` and cite buildNumber/id. Empty/no SR-branch match → `WATCH` (transient, CI still running). |
| `Ship Assessment validation feed` | `maestro_builds` / BAR asset details for the verified SR-branch build | Patch to `READY` only after verifying the SR-branch build has a published `NugetFeed` location for the `Microsoft.Maui.Controls` asset (the per-build `darc-pub-dotnet-maui-<sha8>` endpoint). If the build exists but the `NugetFeed` location is absent/unconfirmed, leave/patch as `WATCH` and tell the captain to wait for BAR asset publishing rather than linking a guessed feed URL. |
| `Milestone hygiene` (API failure) | Re-run `gh auth status` and retry — milestone checks use plain `gh api`, so UNKNOWN means gh isn't scoped right. |

Always cite the MCP query result in your write-up (e.g. *"Verified via `maestro_default_channels`: SR8 is **not** in the mapping list — see darc command above"*).
Expand Down
10 changes: 5 additions & 5 deletions .github/aw/actions-lock.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
{
"entries": {
"actions/checkout@v4": {
"repo": "actions/checkout",
"version": "v4",
"sha": "11d5960a326750d5838078e36cf38b85af677262"
},
"actions/checkout@v7.0.1": {
"repo": "actions/checkout",
"version": "v7.0.1",
Expand All @@ -25,11 +30,6 @@
"version": "v7.0.1",
"sha": "043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"
},
"github/gh-aw-actions/setup-cli@v0.82.14": {
"repo": "github/gh-aw-actions/setup-cli",
"version": "v0.82.14",
"sha": "b6d1443e05b8716267fa19425b99aa4f12006b4a"
},
"github/gh-aw-actions/setup@v0.82.14": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.82.14",
Expand Down
6 changes: 3 additions & 3 deletions .github/docs/agent-labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Manual labels are applied by MAUI maintainers. Queue labels are applied by deter
| Label | Color | Description | Applied When |
|-------|-------|-------------|--------------|
| `s/agent-fix-implemented` | 🟣 `#7B1FA2` | PR author implemented the agent's suggested fix | Maintainer applies when PR author adopts agent's recommendation |
| `s/agent-ready-for-rerun` | 🟣 `#5319E7` | AI review has new PR activity and is ready for rerun | `/review rerun` finds new comments or commits after the latest AI Summary / previous rerun request |
| `s/agent-ready-for-rerun` | 🟣 `#5319E7` | AI review has new PR activity and is ready for rerun | Deterministic rerun automation finds new PR-author comments or commits after the latest AI Summary |
| `s/agent-review-in-progress` | 🟡 `#FBCA04` | AI review is currently running for this PR | Applied before triggering the async AzDO review pipeline and removed by pipeline cleanup; stale locks can be recovered after a conservative timeout |

---
Expand All @@ -72,7 +72,7 @@ Review-PR.ps1
└── Non-fatal: errors warn but don't fail the workflow
```

Most review outcome labels are applied from `Review-PR.ps1` Phase 4. The exceptions are queue/lock labels: `s/agent-ready-for-rerun` is applied by the deterministic `/review rerun` GitHub Action path after checking for new comments or commits, and `s/agent-review-in-progress` is applied before triggering the async AzDO review pipeline. The rerun path does not use AI to decide whether these labels apply. The lock label normally clears in the AzDO cleanup stage; trigger paths treat very old locks as stale so a cancelled pipeline does not permanently block reviews.
Most review outcome labels are applied from `Review-PR.ps1` Phase 4. The exceptions are queue/lock labels: `s/agent-ready-for-rerun` is applied by deterministic automation after checking for new PR-author comments or commits, and `s/agent-review-in-progress` is applied before triggering the async AzDO review pipeline. The queue eligibility check does not use AI. The lock label normally clears in the AzDO cleanup stage; trigger paths treat very old locks as stale so a cancelled pipeline does not permanently block reviews.

### How Labels Are Parsed

Expand Down Expand Up @@ -141,7 +141,7 @@ is:pr label:s/agent-reviewed
|------|---------|
| `.github/scripts/shared/Update-AgentLabels.ps1` | Label helper module (all label logic) |
| `.github/scripts/Review-PR.ps1` | Orchestrator that calls `Apply-AgentLabels` in Phase 4 |
| `.github/scripts/Resolve-RerunEligibility.ps1` | Deterministic `/review rerun` checker that can apply `s/agent-ready-for-rerun` |
| `.github/scripts/Resolve-RerunEligibility.ps1` | Deterministic rerun eligibility and context helpers used by queue automation |
| `.github/scripts/Invoke-RerunReviewTrigger.ps1` | Safe-output handler that validates rerun decisions and emits an actions list; the scanner then dispatches `review-trigger.yml` (which applies `s/agent-review-in-progress` and triggers the AzDO review) |
| `.github/workflows/review-trigger.yml` | Manual `/review` trigger that applies `s/agent-review-in-progress` before triggering AzDO reviews |
| `eng/pipelines/ci-copilot.yml` | AzDO review pipeline that removes `s/agent-review-in-progress` in final cleanup |
Expand Down
15 changes: 15 additions & 0 deletions .github/docs/maui-ci-facts.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,21 @@ Messages like `Baseline snapshot not yet created`, missing snapshot paths, or sn
environment-version mismatches are strong **unrelated** evidence — unless the PR adds or
modifies that visual test or the affected snapshot/platform.

The automated `/review tests` lane gathers failed UI result IDs from the public
`vstmr.dev.azure.com/.../testresults/resultsbybuild` endpoint, then reads the public
result-detail and attachment APIs. It publishes validated baseline/actual/diff PNGs to
the repository's `review-tests-assets` branch. A trusted post-step inserts as many
complete expandable comparison panels as fit inside the single test-failure analysis
comment while enforcing gh-aw's URL, mention, and character limits; excess panels are
reported as omitted rather than creating another comment. Visual publishing is
supplementary evidence only: missing images never raise or lower the deterministic
verdict ceiling. Each panel also shows a conservative relationship label derived from
the exact test-and-platform `deterministicAttribution` plus exact changed snapshot/test
scope: `regressed-vs-base` or directly changed visual coverage is Likely PR-caused,
`pre-existing-on-base` or `known-issue` is Likely unrelated, and indeterminate or
unmatched evidence remains Needs human investigation. A same-named snapshot on another
platform and platform or area mismatch alone never change the label.

## Platform mismatch

Platform mismatch is **supporting** evidence, not proof. An iOS-only test failing on a
Expand Down
54 changes: 12 additions & 42 deletions .github/docs/pr-review-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
This guide explains the automated review commands used in dotnet/maui pull requests:

- `/review`
- `/review rerun`
- `/review tests`

It is intended for Microsoft maintainers and community contributors who want to understand when to request an automated review, what the automation does, and how to interpret the resulting comments.
Expand All @@ -14,22 +13,14 @@ It is intended for Microsoft maintainers and community contributors who want to
| --- | --- | --- | --- |
| `/review` | Repository users with write, maintain, or admin access | Queues the full MAUI Copilot PR review pipeline. | Updates the PR with an `AI Summary` comment. |
| `/review <platform>` | Repository users with write, maintain, or admin access | Queues the full review pipeline for a specific platform: `android`, `ios`, `catalyst`, or `windows`. | Updates the PR with an `AI Summary` comment. |
| `/review rerun` | Repository users with write, maintain, or admin access; or non-first-time PR authors on their own PR | Requests a fresh full review after comments, commits, or CI context changed. Applies the `s/agent-ready-for-rerun` label; the hourly scanner then triggers the review pipeline. | Adds or replaces a review session in the `AI Summary` comment. |
| `/review tests` | Repository users with write, maintain, or admin access | Reviews current CI/test failures and classifies whether they are likely PR-caused, unrelated, or insufficiently evidenced. | Adds or updates a `Test Failure Review` comment. |

Community contributors can trigger `/review rerun` on their own PR after their first contribution. For `/review` and `/review tests`, only repository users with write access can trigger these commands. If you are a first-time contributor, ask a maintainer to run the relevant command for your PR.
Only repository users with write access can trigger these commands. Community contributors should ask a maintainer to run the relevant command for their PR.

## Choosing the right command

Use `/review` when you want the complete automated PR review. This is the normal entry point for maintainers reviewing a PR.

Use `/review rerun` when the PR already has an AI review but something changed enough that the previous review may be stale. Typical reasons:

- the author pushed new commits;
- the author replied to review feedback;
- CI or test results changed;
- a maintainer wants a deterministic fresh review session without manually interpreting older output.

Use `/review tests` when the question is specifically about CI/test failures, for example:

- "Is this failure likely caused by the PR?"
Expand Down Expand Up @@ -72,7 +63,7 @@ The trigger is implemented by `.github/workflows/review-trigger.yml`. It:

The workflow intentionally does not handle `/review tests`; that subcommand is reserved for the test-failure review workflow.

**Note**: Command comments are minimized (collapsed as "Resolved") after authorization to reduce conversation clutter while preserving the comment history for the automated rerun scanner. Unauthorized or malformed command comments remain fully visible.
**Note**: Command comments are minimized (collapsed as "Resolved") after authorization to reduce conversation clutter while preserving the comment history. Unauthorized or malformed command comments remain fully visible.

### Platform inference

Expand Down Expand Up @@ -105,26 +96,16 @@ The PR review script is `.github/scripts/Review-PR.ps1`. It orchestrates the cor

The generated PR comment is a single session-based `AI Summary` comment. New runs replace the review and hide older sessions, keyed by the reviewed commit.

## `/review rerun`: fresh full review

Comment `/review rerun` when you want a new full review session after the PR changed.

Operationally, `/review rerun` applies the `s/agent-ready-for-rerun` label to your PR. The hourly `rerun-review-scanner` workflow checks for eligible PRs with this label and triggers the review pipeline. This means your review may be delayed up to ~1 hour, or may not run if the PR is ineligible.

If you need an immediate review, use `/review` instead (write access required).

### Eligibility requirements
## Automatic fresh reviews

The `/review rerun` workflow checks for **PR author activity** since the latest AI Summary or rerun checkpoint:
After an AI Summary is posted, deterministic rerun automation can queue a fresh full review when it detects new **PR author activity**:

- New commits (head SHA changed)
- New non-command comments from the PR author

**Important**: Reviewer or maintainer reminder comments do NOT satisfy the rerun eligibility check. Only author activity triggers a rerun.
The automation applies the `s/agent-ready-for-rerun` label. An hourly scanner processes queued reruns and triggers them when appropriate. No comment command is required, and identical PR state cannot be re-queued.

When eligible, the workflow applies the `s/agent-ready-for-rerun` label. An automated hourly scanner processes queued reruns and triggers them when appropriate.

### Automated rerun scanner
Reviewer or maintainer reminder comments do not satisfy the eligibility check. If an immediate review is needed, a maintainer can run `/review`.

The repository includes an hourly gh-aw workflow (`.github/workflows/rerun-review-scanner.md`) that:

Expand All @@ -135,17 +116,6 @@ The repository includes an hourly gh-aw workflow (`.github/workflows/rerun-revie

This ensures reruns are processed automatically without manual intervention.

### When to use `/review rerun`

Use it when:

- a previous AI summary is stale;
- the author pushed a fix after review feedback;
- the previous run analyzed the wrong commit or incomplete context;
- a maintainer wants to replace an older session with a fresh one for the current PR head.

Avoid using it repeatedly without new commits or author comments. It consumes CI and agent capacity, and repeated identical runs are unlikely to add useful information.

## `/review tests`: test-failure review

### Trigger
Expand Down Expand Up @@ -234,7 +204,7 @@ Do not treat `Insufficient data` as "unrelated." It means a human or a rerun wit

### AI Summary

The full `/review` and `/review rerun` pipeline posts an `AI Summary` comment. It may include:
The full `/review` pipeline and automatic reruns post an `AI Summary` comment. It may include:

- gate status;
- UI test results;
Expand Down Expand Up @@ -264,7 +234,7 @@ The verdict details and "Test Failure Review" label live in badges and in the ex
2. Run `/review` when a PR is ready for automated review.
3. Read the `AI Summary` comment and check whether the review found actionable issues.
4. If CI is red or ambiguous, run `/review tests` to get a focused failure-causality report.
5. If the author pushes fixes or comments materially change the context, run `/review rerun`.
5. If the author pushes fixes or adds material context, allow the automatic rerun queue to process it or run `/review` for an immediate review.
6. Use human judgment for merge decisions. These workflows provide evidence and recommendations, not final approval authority.

## Recommended workflow for community contributors
Expand All @@ -274,7 +244,7 @@ The verdict details and "Test Failure Review" label live in badges and in the ex
3. If you need an automated review, ask a maintainer to run `/review`.
4. If CI is red and you are unsure whether it is caused by your changes, ask a maintainer to run `/review tests`.
5. When an automated comment is posted, read the summary first, then expand evidence sections for details.
6. Push fixes or reply with clarifying information, then ask a maintainer whether `/review rerun` is useful.
6. Push fixes or reply with clarifying information; eligible author activity is detected automatically for a fresh review.

## Safety and trust boundaries

Expand All @@ -284,6 +254,7 @@ Important safeguards:

- `/review` requires repository write-level permissions and queues a trusted AzDO pipeline.
- `/review tests` is comment-only and uses gh-aw safe outputs for PR comments.
- Automatic reruns require deterministic new PR-author activity before the queue label is applied.
- The full review pipeline keeps PR-controlled code separated from trusted scripts where possible.
- Review comments should be treated as assistant-generated evidence, not as a substitute for human review.

Expand All @@ -294,9 +265,8 @@ Important safeguards:
| `/review` does nothing | The commenter does not have write/maintain/admin access, or the comment is not on a PR. | Ask a maintainer to run the command on the PR. |
| `/review` used the wrong platform | Platform labels were missing or ambiguous. | Re-run with an explicit platform, for example `/review ios`. |
| `/review tests` says `Insufficient data` | Build/log/Helix evidence was inaccessible or incomplete. | Re-run later, provide a build ID, or run locally with Azure CLI/AzDO auth. |
| The AI Summary looks stale | New commits or comments landed after the last review. | Run `/review rerun`. |
| The AI Summary looks stale | New commits or author comments landed after the last review. | Wait for the automatic rerun queue, or ask a maintainer to run `/review` for an immediate review. |
| There are multiple old AI Summary comments | Each comment holds only the latest session (keyed to its HEAD commit); previous review comments are minimized and hidden as outdated. | Expand the **Review Sessions** section in the newest comment — it reflects the current HEAD commit. |
| `/review rerun` didn't trigger | Only PR author activity (commits or non-command comments) satisfies eligibility. Reviewer comments don't trigger reruns. | Wait for author activity or use `/review` to force a new review. |
| Command comment is still visible | The commenter may lack authorization, or the command was malformed. | Check actor permissions and command syntax. Authorized commands are minimized after processing. |

## Related files
Expand All @@ -306,7 +276,7 @@ Important safeguards:
- `.github/scripts/Review-PR.ps1` — local script orchestrating full PR review phases.
- `.github/scripts/post-ai-summary-comment.ps1` — AI Summary comment formatter.
- `.github/workflows/copilot-review-tests.md` — gh-aw source for `/review tests`.
- `.github/workflows/rerun-review-scanner.md` — gh-aw hourly scanner for queued `/review rerun` requests.
- `.github/workflows/rerun-review-scanner.md` — gh-aw hourly scanner for automatically queued fresh reviews.
- `.github/scripts/Resolve-RerunEligibility.ps1` — determines if a PR is eligible for rerun based on author activity.
- `.github/scripts/Query-RerunReadyPRs.ps1` — queries PRs labeled `s/agent-ready-for-rerun`.
- `.github/skills/review-test-failures/SKILL.md` — classification rubric for test-failure reviews.
Expand Down
8 changes: 7 additions & 1 deletion .github/scripts/Post-AISummaryComment.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

BeforeAll {
$scriptPath = Join-Path $PSScriptRoot 'post-ai-summary-comment.ps1'
$script:ScriptSource = Get-Content -Raw -LiteralPath $scriptPath
$tokens = $null
$parseErrors = $null
$ast = [System.Management.Automation.Language.Parser]::ParseFile($scriptPath, [ref]$tokens, [ref]$parseErrors)
Expand Down Expand Up @@ -127,11 +128,16 @@ Describe 'Add-MissingUITestResultsNote' {
It 'annotates a bare "detected categories" placeholder with no results' {
$result = Add-MissingUITestResultsNote -Content '**Detected UI test categories:** `Picker,ViewBaseTests`'
$result | Should -Match 'No UI test results were produced'
$result | Should -Match '/review rerun'
$result | Should -Match 'Fix the build/gate issues and push again'
$result | Should -Not -Match '/review rerun'
# Original content is preserved.
$result | Should -Match 'Detected UI test categories'
}

It 'does not advertise the removed rerun command' {
$script:ScriptSource | Should -Not -Match '/review rerun'
}

It 'does NOT annotate when deep results are present' {
$content = "**Detected UI test categories:** ``Picker``" + [Environment]::NewLine +
'✅ **Deep UI tests** — 12 passed, 0 failed across 1 category on platform-pool agent.'
Expand Down
Loading
Loading