Skip to content

Commit 3e78a6a

Browse files
cortisikochristopherferreira9javiergarciavera
authored
test: unified framework main branch (#27939)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until the template has been completely filled out, and PR status checks have passed at least once. --> ## **Description** > **Migrates performance testing from `appwright` to `playwright` and modernizes the Playwright/Appium test harness.** The GitHub workflow and `package.json` scripts are switched to `run-playwright:*`, and several performance specs are rewritten from legacy WDIO screen-objects to new page-objects/flows. > > **Adds new Playwright fixtures and utilities for performance runs.** A `currentDeviceDetails` fixture is introduced, plus a `performanceTracker` fixture that auto-attaches metrics post-test, validates quality gates (skipping retries after threshold-only failures), publishes scenarios to Sentry, and stores session metadata. > > **Improves Appium interaction reliability and provider integration.** Playwright matchers/gestures gain stability-aware `waitAndTap`, better text/XPath handling, app lifecycle helpers (terminate/activate/background/hide keyboard), standardized default timeouts, and BrowserStack/emulator capability updates including per-platform app identifiers and a provider `getRecordingUrl` for video links. > > **Updates test IDs and selectors used by E2E.** Predict market details testIDs are normalized (tab vs tab-content), and confirmations add a `KEYBOARD_CONTAINER` id with corresponding page-object updates. > ## **Changelog** <!-- If this PR is not End-User-Facing and should not show up in the CHANGELOG, you can choose to either: 1. Write `CHANGELOG entry: null` 2. Label with `no-changelog` If this PR is End-User-Facing, please write a short User-Facing description in the past tense like: `CHANGELOG entry: Added a new tab for users to see their NFTs` `CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker` (This helps the Release Engineer do their job more quickly and accurately) --> CHANGELOG entry: ## **Related issues** Fixes: https://consensyssoftware.atlassian.net/browse/MMQA-1454 ## **Manual testing steps** N/A ## **Screenshots/Recordings** <!-- If applicable, add screenshots and/or recordings to visualize the before and after of your change. --> ### **Before** N/A <!-- [screenshots/recordings] --> ### **After** N/A <!-- [screenshots/recordings] --> ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I've included tests if applicable - [x] I've documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I've applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Broad changes across the E2E/performance harness (fixtures, matchers/gestures, BrowserStack capabilities, and multiple rewritten specs) could introduce new test flakiness or CI/run-time regressions despite being non-production code. > > **Overview** > Migrates BrowserStack performance test execution from `appwright` to `playwright` by switching the GitHub workflow and `package.json` scripts to `run-playwright:*` commands. > > Refactors the Playwright/Appium test harness with **new fixtures** (including `currentDeviceDetails` and an auto-managed `performanceTracker` that attaches metrics, enforces quality gates, and publishes to Sentry), plus reliability improvements to matchers/gestures (stability-aware `waitAndTap`, richer XPath/text handling, app lifecycle helpers, and standardized default timeouts). > > Rewrites several performance specs and page objects to the unified page-object/flow style (wallet/login, swaps, perps), adds a new `perps.flow` helper, and updates/normalizes test IDs used by E2E (Predict market details tab/content IDs and a new confirmations `KEYBOARD_CONTAINER` selector). > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 7a072f2. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Christopher Ferreira <christopher.ferreira@consensys.net> Co-authored-by: Christopher Ferreira <104831203+christopherferreira9@users.noreply.github.com> Co-authored-by: Javier Garcia Vera <javier.vera@consensys.net>
1 parent a7980e3 commit 3e78a6a

101 files changed

Lines changed: 4088 additions & 2044 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/performance-test-runner.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -241,11 +241,11 @@ jobs:
241241
242242
# Run the appropriate test command based on build_type flag
243243
if [ "${{ inputs.build_type }}" = "onboarding" ]; then
244-
yarn run-appwright:${{ inputs.platform }}-onboarding-bs
244+
yarn run-playwright:${{ inputs.platform }}-onboarding-bs
245245
elif [ "${{ inputs.build_type }}" = "mm-connect" ]; then
246-
yarn run-appwright:mm-connect-${{ inputs.platform }}-bs
246+
yarn run-playwright:mm-connect-${{ inputs.platform }}-bs
247247
else
248-
yarn run-appwright:${{ inputs.platform }}-bs
248+
yarn run-playwright:${{ inputs.platform }}-bs
249249
fi
250250
251251
echo "✅ ${{ inputs.build_type }} tests completed for ${{ inputs.platform }} on ${{ matrix.device.name }}"

app/components/UI/Predict/Predict.testIds.ts

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -82,9 +82,14 @@ export const getPredictFeedMockSelector = {
8282

8383
export type PredictMarketDetailsTabKey = 'positions' | 'outcomes' | 'about';
8484

85+
const PREDICT_MARKET_DETAILS_SELECTOR_BASE = 'predict-market-details';
86+
8587
export const getPredictMarketDetailsSelector = {
88+
tabBar: `${PREDICT_MARKET_DETAILS_SELECTOR_BASE}-tab-bar`,
89+
tabContent: (tabKey: PredictMarketDetailsTabKey) =>
90+
`${PREDICT_MARKET_DETAILS_SELECTOR_BASE}-${tabKey}-tab-content`,
8691
tabBarTab: (tabKey: PredictMarketDetailsTabKey) =>
87-
`predict-market-details-tab-bar-tab-${tabKey}`,
92+
`${PREDICT_MARKET_DETAILS_SELECTOR_BASE}-tab-bar-tab-${tabKey}`,
8893
icon: (name: string) => `icon-${name}`,
8994
} as const;
9095

@@ -98,20 +103,16 @@ export const PredictMarketDetailsSelectorsIDs = {
98103
SHARE_BUTTON: 'predict-market-details-share-button',
99104

100105
// Tabs
101-
TAB_BAR: 'predict-market-details-tab-bar',
102-
ABOUT_TAB: 'predict-market-details-about-tab',
103-
POSITIONS_TAB: 'predict-market-details-positions-tab',
104-
OUTCOMES_TAB: 'predict-market-details-outcomes-tab',
105-
106-
// Tab labels
107-
POSITIONS_TAB_LABEL: getPredictMarketDetailsSelector.tabBarTab('positions'),
108-
OUTCOMES_TAB_LABEL: getPredictMarketDetailsSelector.tabBarTab('outcomes'),
109-
ABOUT_TAB_LABEL: getPredictMarketDetailsSelector.tabBarTab('about'),
106+
TAB_BAR: getPredictMarketDetailsSelector.tabBar,
107+
ABOUT_TAB: getPredictMarketDetailsSelector.tabBarTab('about'),
108+
POSITIONS_TAB: getPredictMarketDetailsSelector.tabBarTab('positions'),
109+
OUTCOMES_TAB: getPredictMarketDetailsSelector.tabBarTab('outcomes'),
110110

111111
// Tab content containers
112-
ABOUT_TAB_CONTENT: 'about-tab-content',
113-
POSITIONS_TAB_CONTENT: 'positions-tab-content',
114-
OUTCOMES_TAB_CONTENT: 'outcomes-tab-content',
112+
ABOUT_TAB_CONTENT: getPredictMarketDetailsSelector.tabContent('about'),
113+
POSITIONS_TAB_CONTENT:
114+
getPredictMarketDetailsSelector.tabContent('positions'),
115+
OUTCOMES_TAB_CONTENT: getPredictMarketDetailsSelector.tabContent('outcomes'),
115116
MARKET_DETAILS_CASH_OUT_BUTTON: 'predict-market-details-cash-out-button',
116117
CLAIM_WINNINGS_BUTTON: 'predict-market-details-claim-winnings-button',
117118

app/components/UI/Predict/views/PredictMarketDetails/components/PredictMarketDetailsTabContent.tsx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ const PredictMarketDetailsTabContent = memo(
6868
return (
6969
<Box
7070
twClassName="px-3 pt-4 pb-8"
71-
testID={PredictMarketDetailsSelectorsIDs.ABOUT_TAB}
71+
testID={PredictMarketDetailsSelectorsIDs.ABOUT_TAB_CONTENT}
7272
>
7373
<PredictMarketDetailsAbout
7474
market={market}
@@ -81,7 +81,7 @@ const PredictMarketDetailsTabContent = memo(
8181
return (
8282
<Box
8383
twClassName="px-3 pt-4 pb-8"
84-
testID={PredictMarketDetailsSelectorsIDs.POSITIONS_TAB}
84+
testID={PredictMarketDetailsSelectorsIDs.POSITIONS_TAB_CONTENT}
8585
>
8686
<PredictMarketDetailsPositions
8787
activePositions={activePositions}
@@ -95,7 +95,7 @@ const PredictMarketDetailsTabContent = memo(
9595
return (
9696
<Box
9797
twClassName="px-3 pt-4 pb-8"
98-
testID={PredictMarketDetailsSelectorsIDs.OUTCOMES_TAB}
98+
testID={PredictMarketDetailsSelectorsIDs.OUTCOMES_TAB_CONTENT}
9999
>
100100
<PredictMarketDetailsOutcomes
101101
market={market}

app/components/Views/confirmations/ConfirmationView.testIds.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,4 +101,5 @@ export const TransactionPayComponentIDs = {
101101
PAY_WITH_FIAT: 'pay-with-fiat',
102102
PAY_WITH_SYMBOL: 'pay-with-symbol',
103103
PAY_WITH_TOKEN_LIST: 'pay-with-token-list',
104+
KEYBOARD_CONTAINER: 'custom-amount-input',
104105
};

package.json

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -110,15 +110,17 @@
110110
"test:reassure:branch": "REASSURE=true yarn reassure --branch && node -e \"const r=require('./.reassure/output.json'); if ((r.significant||[]).length>0) { console.error('Reassure: significant regressions detected'); process.exit(1);} else { process.exit(0);} \"",
111111
"test:e2e:feature-flag:coverage": "ts-node tests/feature-flags/feature-flag-coverage-report.ts",
112112
"test:e2e:ai-test-plan": "node -r esbuild-register tests/tools/e2e-ai-analyzer --mode generate-test-plan --auto-ff",
113-
"run-appwright:android-bs": "yarn appwright test --project browserstack-android --config tests/appwright.config.ts",
114-
"run-appwright:android-onboarding-bs": "yarn appwright test --project android-onboarding --project android-onboarding-seedless --config tests/appwright.config.ts",
115-
"run-appwright:ios-onboarding-bs": "yarn appwright test --project ios-onboarding --project ios-onboarding-seedless --config tests/appwright.config.ts",
116-
"run-appwright:ios-bs": "yarn appwright test --project browserstack-ios --config tests/appwright.config.ts",
117-
"run-appwright:android": "yarn appwright test --project android --config tests/appwright.config.ts",
118-
"run-appwright:ios": "yarn appwright test --project ios --config tests/appwright.config.ts",
119-
"run-appwright:mm-connect-android-local": "yarn appwright test --project mm-connect-android-local --config tests/appwright.config.ts",
120-
"run-appwright:mm-connect-android-bs": "yarn appwright test --project mm-connect-android-browserstack --config tests/appwright.config.ts",
121-
"run-appwright:mm-connect-android-bs-local": "yarn appwright test --project mm-connect-android-browserstack --config tests/appwright.config.ts",
113+
"run-playwright:android-bs": "yarn playwright test --project browserstack-android --config tests/playwright.config.ts",
114+
"run-playwright:android-onboarding-bs": "yarn playwright test --project android-onboarding --project android-onboarding-seedless --config tests/playwright.config.ts",
115+
"run-playwright:ios-onboarding-bs": "yarn playwright test --project ios-onboarding --project ios-onboarding-seedless --config tests/playwright.config.ts",
116+
"run-playwright:android-seedless-bs": "yarn playwright test --project android-onboarding-seedless --config tests/playwright.config.ts",
117+
"run-playwright:ios-seedless-bs": "yarn playwright test --project ios-onboarding-seedless --config tests/playwright.config.ts",
118+
"run-playwright:ios-bs": "yarn playwright test --project browserstack-ios --config tests/playwright.config.ts",
119+
"run-playwright:android": "yarn playwright test --project android --config tests/playwright.config.ts",
120+
"run-playwright:ios": "yarn playwright test --project ios --config tests/playwright.config.ts",
121+
"run-playwright:mm-connect-android-local": "yarn playwright test --project mm-connect-android-local --config tests/playwright.config.ts",
122+
"run-playwright:mm-connect-android-bs": "yarn playwright test --project mm-connect-android-browserstack --config tests/playwright.config.ts",
123+
"run-playwright:mm-connect-android-bs-local": "yarn playwright test --project mm-connect-android-browserstack --config tests/playwright.config.ts",
122124
"test:depcheck": "yarn depcheck",
123125
"test:tgz-check": "./scripts/tgz-check.sh",
124126
"test:attribution-check": "./scripts/attributions-check.sh",

tests/flows/perps.flow.ts

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
import {
2+
asDetoxElement,
3+
asPlaywrightElement,
4+
Assertions,
5+
encapsulatedAction,
6+
} from '../framework';
7+
import PerpsMarketDetailsView from '../page-objects/Perps/PerpsMarketDetailsView';
8+
import PerpsOrderView from '../page-objects/Perps/PerpsOrderView';
9+
10+
/**
11+
* Checks if the position is open by checking if the close button is visible.
12+
* @returns {Promise<boolean>} True if the position is open, false otherwise.
13+
*/
14+
export const isPositionOpen = async (timeout = 5000): Promise<boolean> => {
15+
let isPositionOpen = false;
16+
await encapsulatedAction({
17+
detox: async () => {
18+
try {
19+
const el = asDetoxElement(PerpsMarketDetailsView.closeButton);
20+
await Assertions.expectElementToBeVisible(el, {
21+
timeout,
22+
description: 'Close position button',
23+
});
24+
isPositionOpen = true;
25+
} catch {
26+
isPositionOpen = false;
27+
}
28+
},
29+
appium: async () => {
30+
try {
31+
const closeEl = await asPlaywrightElement(
32+
PerpsMarketDetailsView.closeButton,
33+
);
34+
isPositionOpen = await closeEl.isVisible();
35+
} catch {
36+
// Element lookup timed out — position is not open
37+
isPositionOpen = false;
38+
}
39+
},
40+
});
41+
return isPositionOpen;
42+
};
43+
44+
export const waitForPositionOpen = async (
45+
timeout = 20000,
46+
interval = 1000,
47+
): Promise<void> => {
48+
const start = Date.now();
49+
while (Date.now() - start < timeout) {
50+
if (await isPositionOpen()) return;
51+
await new Promise((resolve) => setTimeout(resolve, interval));
52+
}
53+
throw new Error(`Position not open after ${timeout}ms`);
54+
};
55+
56+
/**
57+
* Checks if the place order button is visible.
58+
* @returns {Promise<boolean>} True if the place order button is visible, false otherwise.
59+
*/
60+
export const isPlaceOrderButtonVisible = async (): Promise<boolean> => {
61+
let visible = false;
62+
await encapsulatedAction({
63+
detox: async () => {
64+
try {
65+
const el = asDetoxElement(PerpsOrderView.placeOrderButton);
66+
await Assertions.expectElementToBeVisible(el, {
67+
timeout: 5000,
68+
description: 'Place order button',
69+
});
70+
visible = true;
71+
} catch {
72+
visible = false;
73+
}
74+
},
75+
appium: async () => {
76+
try {
77+
const placeOrderButtonEl = await asPlaywrightElement(
78+
PerpsOrderView.placeOrderButton,
79+
);
80+
visible = await placeOrderButtonEl.isVisible();
81+
} catch {
82+
visible = false;
83+
}
84+
},
85+
});
86+
return visible;
87+
};
88+
89+
/**
90+
* Waits for the order screen to be visible.
91+
* @param timeout - The timeout in milliseconds.
92+
* @param interval - The interval in milliseconds.
93+
* @returns {Promise<void>} Resolves when the order screen is visible.
94+
*/
95+
export const waitForOrderScreenVisible = async (
96+
timeout = 20000,
97+
interval = 1000,
98+
): Promise<void> => {
99+
const start = Date.now();
100+
while (Date.now() - start < timeout) {
101+
if (await isPlaceOrderButtonVisible()) return;
102+
await new Promise((resolve) => setTimeout(resolve, interval));
103+
}
104+
throw new Error(`Order screen not visible after ${timeout}ms`);
105+
};

0 commit comments

Comments
 (0)