Commit d0a4402
chore(runway): cherry-pick test: integrate AI test plan into RC build comments (#29185)
- test: integrate AI test plan into RC build comments (#28951)
<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until the template has been
completely filled out, and PR status checks have passed at least once.
-->
## **Description**
<!--
Write a short description of the changes included in this pull request,
also include relevant motivation and context. Have in mind the following
questions:
1. What is the reason for the change?
2. What is the improvement/solution?
-->
Integrates AI test plan generation directly into RC build comments
Changes
- New scripts/build-announce/ module (TypeScript):
- index.ts - Main orchestrator (replaces post-rc-build-comment.mjs)
- test-plan-section.ts - Converts test plan JSON to markdown with
collapsible sections
- types.ts - TypeScript interfaces
- utils.ts - Helper functions (GitHub API, URL validation)
- Updated build-rc-auto.yml:
- Generates test plan inline during RC build workflow
- Removed label requirement (fully automatic)
- Uploads JSON to gh-pages for automation
- Deleted:
- generate-rc-test-plan.yml (no longer separate workflow)
- post-rc-build-comment.mjs (replaced by TypeScript)
What it does
- Stats table (risk score, high/medium risk counts, files changed, teams
signed off)
- Collapsible sections for executive summary, high/medium risk scenarios
- Teams sign-off status section
- Excluded features (feature flags) section
- JSON link at bottom for automation
- Graceful fallback if test plan generation fails
## **Changelog**
<!--
If this PR is not End-User-Facing and should not show up in the
CHANGELOG, you can choose to either:
1. Write `CHANGELOG entry: null`
2. Label with `no-changelog`
If this PR is End-User-Facing, please write a short User-Facing
description in the past tense like:
`CHANGELOG entry: Added a new tab for users to see their NFTs`
`CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker`
(This helps the Release Engineer do their job more quickly and
accurately)
-->
CHANGELOG entry: null
## **Related issues**
Fixes: https://consensyssoftware.atlassian.net/browse/MCRM-69
## **Manual testing steps**
```gherkin
Feature: my feature name
Scenario: user [verb for user action]
Given [describe expected initial app state]
When user [verb for user action]
Then [describe expected outcome]
```
## **Screenshots/Recordings**
<!-- If applicable, add screenshots and/or recordings to visualize the
before and after of your change. -->
### **Before**
<!-- [screenshots/recordings] -->
#28355 (comment)
<img width="608" height="421" alt="Screenshot 2026-04-16 at 5 38 08 PM"
src="https://github.com/user-attachments/assets/2d211150-9e2e-4ac7-8f71-f74218989159"
/>
### **After**
Should match Extension RC PR comment:
MetaMask/metamask-extension#41626 (comment)
<!-- [screenshots/recordings] -->
## **Pre-merge author checklist**
- [ ] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile
Coding
Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md).
- [ ] I've completed the PR template to the best of my ability
- [ ] I've included tests if applicable
- [ ] I've documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [ ] I've applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.
#### Performance checks (if applicable)
- [ ] I've tested on Android
- Ideally on a mid-range device; emulator is acceptable
- [ ] I've tested with a power user scenario
- Use these [power-user
SRPs](https://consensyssoftware.atlassian.net/wiki/spaces/TL1/pages/edit-v2/401401446401?draftShareId=9d77e1e1-4bdc-4be1-9ebb-ccd916988d93)
to import wallets with many accounts and tokens
- [ ] I've instrumented key operations with Sentry traces for production
performance metrics
- See [`trace()`](/app/util/trace.ts) for usage and
[`addToken`](/app/components/Views/AddAsset/components/AddCustomToken/AddCustomToken.tsx#L274)
for an example
For performance guidelines and tooling, see the [Performance
Guide](https://consensyssoftware.atlassian.net/wiki/spaces/TL1/pages/400085549067/Performance+Guide+for+Engineers).
## **Pre-merge reviewer checklist**
- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Changes the release CI workflow behavior and permissions
(auto-triggering RC builds, generating content via external AI
providers, and pushing to `gh-pages`), which could affect release
pipeline reliability and repository writes if misconfigured.
>
> **Overview**
> **RC builds are now fully automatic for release branches with an open
PR.** The `build-rc-auto.yml` workflow no longer gates builds on an
`auto-rc-builds` label; instead it finds the associated PR and proceeds
when one exists.
>
> **RC build announcements now optionally include an AI-generated test
plan inline.** The workflow replaces `scripts/post-rc-build-comment.mjs`
with a new TypeScript `scripts/build-announce/` runner that posts the
build links, generates/parses `release-test-plan.json` via
`tests/tools/e2e-ai-analyzer` when AI provider keys are present, formats
results into collapsible markdown sections, and minimizes older RC
announce comments.
>
> **Test plan hosting is consolidated into the RC build workflow.** The
standalone `generate-rc-test-plan.yml` comment-triggered workflow is
removed, and the RC workflow now attempts (best-effort) to commit the
generated JSON to the `gh-pages` branch under `test-plans/` for later
automation/links.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
3f0ea37. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
[84dc598](84dc598)
Co-authored-by: sleepytanya <104780023+sleepytanya@users.noreply.github.com>1 parent 0fa8ee4 commit d0a4402
7 files changed
Lines changed: 842 additions & 606 deletions
File tree
- .github/workflows
- scripts
- build-announce
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
5 | | - | |
| 4 | + | |
6 | 5 | | |
7 | 6 | | |
8 | 7 | | |
9 | | - | |
| 8 | + | |
10 | 9 | | |
11 | 10 | | |
12 | 11 | | |
13 | 12 | | |
| 13 | + | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
33 | | - | |
34 | | - | |
| 34 | + | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
38 | | - | |
| 39 | + | |
39 | 40 | | |
40 | | - | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| |||
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
67 | | - | |
68 | | - | |
| 68 | + | |
| 69 | + | |
69 | 70 | | |
70 | 71 | | |
71 | 72 | | |
| |||
75 | 76 | | |
76 | 77 | | |
77 | 78 | | |
78 | | - | |
| 79 | + | |
79 | 80 | | |
80 | 81 | | |
81 | 82 | | |
82 | | - | |
| 83 | + | |
83 | 84 | | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
| 85 | + | |
94 | 86 | | |
95 | 87 | | |
96 | 88 | | |
97 | 89 | | |
98 | 90 | | |
99 | 91 | | |
100 | | - | |
101 | | - | |
| 92 | + | |
| 93 | + | |
102 | 94 | | |
103 | 95 | | |
104 | 96 | | |
105 | 97 | | |
106 | | - | |
| 98 | + | |
107 | 99 | | |
108 | 100 | | |
109 | 101 | | |
110 | 102 | | |
111 | 103 | | |
112 | 104 | | |
113 | 105 | | |
114 | | - | |
| 106 | + | |
115 | 107 | | |
116 | | - | |
| 108 | + | |
117 | 109 | | |
118 | 110 | | |
119 | 111 | | |
| |||
124 | 116 | | |
125 | 117 | | |
126 | 118 | | |
127 | | - | |
| 119 | + | |
128 | 120 | | |
129 | | - | |
| 121 | + | |
130 | 122 | | |
131 | 123 | | |
132 | 124 | | |
133 | 125 | | |
134 | 126 | | |
135 | 127 | | |
136 | 128 | | |
137 | | - | |
| 129 | + | |
138 | 130 | | |
139 | 131 | | |
140 | | - | |
| 132 | + | |
141 | 133 | | |
142 | 134 | | |
143 | | - | |
| 135 | + | |
| 136 | + | |
144 | 137 | | |
145 | 138 | | |
| 139 | + | |
146 | 140 | | |
147 | 141 | | |
148 | 142 | | |
149 | | - | |
150 | | - | |
| 143 | + | |
| 144 | + | |
151 | 145 | | |
152 | 146 | | |
153 | 147 | | |
154 | 148 | | |
155 | 149 | | |
156 | 150 | | |
157 | | - | |
158 | | - | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
159 | 154 | | |
160 | 155 | | |
161 | 156 | | |
162 | | - | |
163 | | - | |
| 157 | + | |
| 158 | + | |
164 | 159 | | |
165 | 160 | | |
166 | 161 | | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
167 | 207 | | |
168 | 208 | | |
169 | 209 | | |
170 | 210 | | |
171 | | - | |
| 211 | + | |
172 | 212 | | |
173 | 213 | | |
174 | 214 | | |
175 | 215 | | |
176 | 216 | | |
177 | | - | |
| 217 | + | |
178 | 218 | | |
179 | 219 | | |
180 | 220 | | |
| |||
0 commit comments