Commit 69ab778
authored
test: add nightly run flow for system tests (#29484)
<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until this PR meets the canonical
Definition of Ready For Review in `docs/readme/ready-for-review.md`.
In short: the template must be materially complete (not just section
titles
present), all status checks must be currently passing, and the only
expected
follow-up commits must be reviewer-driven.
-->
## **Description**
Adds a new Nightly System Tests GitHub Actions workflow
(run-system-tests.yml) that runs on a daily cron or manually, optionally
accepting pre-built BrowserStack app URLs.
The workflow builds two RC variants (clean + with pre-imported SRP),
uploads resulting APK/IPA artifacts to BrowserStack, runs Android/iOS
login and onboarding test suites via BrowserStack Local, publishes test
report artifacts, and fails the run if any platform flow fails.
## **Changelog**
<!--
If this PR is not End-User-Facing and should not show up in the
CHANGELOG, you can choose to either:
1. Write `CHANGELOG entry: null`
2. Label with `no-changelog`
If this PR is End-User-Facing, please write a short User-Facing
description in the past tense like:
`CHANGELOG entry: Added a new tab for users to see their NFTs`
`CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker`
(This helps the Release Engineer do their job more quickly and
accurately)
-->
CHANGELOG entry:
## **Related issues**
Fixes:
## **Manual testing steps**
```gherkin
Feature: my feature name
Scenario: user [verb for user action]
Given [describe expected initial app state]
When user [verb for user action]
Then [describe expected outcome]
```
## **Screenshots/Recordings**
<!-- If applicable, add screenshots and/or recordings to visualize the
before and after of your change. -->
### **Before**
<!-- [screenshots/recordings] -->
### **After**
<!-- [screenshots/recordings] -->
## **Pre-merge author checklist**
<!--
Every checklist item must be consciously assessed before marking this PR
as
"Ready for review". A checked box means you deliberately considered that
responsibility, not that you literally performed every action listed.
Unchecked boxes are ambiguous: they are not an implicit "N/A" and they
are not
a silent "skip". See `docs/readme/ready-for-review.md` for the full
checklist
semantics.
-->
- [ ] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile
Coding
Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md).
- [ ] I've completed the PR template to the best of my ability
- [ ] I've included tests if applicable
- [ ] I've documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [ ] I've applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.
#### Performance checks (if applicable)
- [ ] I've tested on Android
- Ideally on a mid-range device; emulator is acceptable
- [ ] I've tested with a power user scenario
- Use these [power-user
SRPs](https://consensyssoftware.atlassian.net/wiki/spaces/TL1/pages/edit-v2/401401446401?draftShareId=9d77e1e1-4bdc-4be1-9ebb-ccd916988d93)
to import wallets with many accounts and tokens
- [ ] I've instrumented key operations with Sentry traces for production
performance metrics
- See [`trace()`](/app/util/trace.ts) for usage and
[`addToken`](/app/components/Views/AddAsset/components/AddCustomToken/AddCustomToken.tsx#L274)
for an example
For performance guidelines and tooling, see the [Performance
Guide](https://consensyssoftware.atlassian.net/wiki/spaces/TL1/pages/400085549067/Performance+Guide+for+Engineers).
## **Pre-merge reviewer checklist**
<!--
Reviewer checklist items follow the same semantics as the author
checklist: an
unchecked box is ambiguous, a checked box means the reviewer consciously
assessed that responsibility. See `docs/readme/ready-for-review.md`.
-->
- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Adds a new scheduled GitHub Actions workflow that builds and uploads
signed RC artifacts and runs device tests on BrowserStack, which may
affect CI load and secret usage if misconfigured.
>
> **Overview**
> Adds a new `Nightly System Tests` GitHub Actions workflow
(`.github/workflows/run-system-tests.yml`) that runs on a daily cron or
manually, builds *clean* and *with-SRP* RC variants, uploads APK/IPA
artifacts to BrowserStack, and executes Android/iOS login + onboarding
system test suites via BrowserStack Local with report artifacts and a
failing summary gate.
>
> Extends `builds.yml` with a new `main-rc-with-srp` build variant (RC
config plus injected `ADDITIONAL_SRP_1` and `PREDEFINED_PASSWORD`) to
support login-focused system tests, and updates `.github/CODEOWNERS` to
assign QA ownership for the new workflow.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
809f391. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->1 parent c8800ec commit 69ab778
3 files changed
Lines changed: 656 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
358 | 358 | | |
359 | 359 | | |
360 | 360 | | |
| 361 | + | |
361 | 362 | | |
362 | 363 | | |
363 | 364 | | |
| |||
0 commit comments