test(MMQA-1340): fixed perps and predictions tests (#25372)

<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until the template has been
completely filled out, and PR status checks have passed at least once.
-->

## **Description**
This PR introduces smart retry logic for performance tests. When a test
fails due to Quality Gates (performance threshold exceeded), the test
will not be retried since the measurement was valid - only the threshold
was exceeded. However, if a test fails due to execution errors (element
not found, timeout, crash), it will retry as expected.
Why?
Retrying a test that exceeded performance thresholds wastes CI resources
and time
The performance measurement was successful; retrying won't change the
result
Tests failing due to UI issues should still retry to handle flakiness
Solution:
Created QualityGateError class to distinguish threshold failures from
other errors
Implemented file-based registry to track quality gate failures across
Playwright workers
Modified performance-test.js fixture to skip retries when previous
attempt failed due to quality gates

<!--
Write a short description of the changes included in this pull request,
also include relevant motivation and context. Have in mind the following
questions:
1. What is the reason for the change?
2. What is the improvement/solution?
-->

## **Changelog**

<!--
If this PR is not End-User-Facing and should not show up in the
CHANGELOG, you can choose to either:
1. Write `CHANGELOG entry: null`
2. Label with `no-changelog`

If this PR is End-User-Facing, please write a short User-Facing
description in the past tense like:
`CHANGELOG entry: Added a new tab for users to see their NFTs`
`CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker`

(This helps the Release Engineer do their job more quickly and
accurately)
-->

CHANGELOG entry:

## **Related issues**

Fixes:

## **Manual testing steps**

```gherkin
Feature: my feature name

  Scenario: user [verb for user action]
    Given [describe expected initial app state]

    When user [verb for user action]
    Then [describe expected outcome]
```

## **Screenshots/Recordings**

<!-- If applicable, add screenshots and/or recordings to visualize the
before and after of your change. -->

### **Before**

<!-- [screenshots/recordings] -->

### **After**

<!-- [screenshots/recordings] -->

## **Pre-merge author checklist**

- [ ] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile
Coding
Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md).
- [ ] I've completed the PR template to the best of my ability
- [ ] I've included tests if applicable
- [ ] I've documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [ ] I've applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.

## **Pre-merge reviewer checklist**

- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.


<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Medium risk because it changes performance test execution semantics
(retry/skip behavior) across workers and refactors several UI
flows/selectors, which could inadvertently skip legitimate retries or
introduce new flakiness if IDs/assumptions are off.
> 
> **Overview**
> **Performance tests now avoid wasting retries on threshold failures.**
A new `QualityGateError` plus a file-backed registry
(`QualityGateError.js`) tracks tests that fail *only* quality gates, and
`performance-test.js` skips subsequent retries for those tests while
still retrying real execution failures.
> 
> **Reporting and quality gate plumbing updated.**
`QualityGatesValidator.assertThresholds` now throws `QualityGateError`,
and `custom-reporter.js` clears any persisted quality-gate failures at
run start.
> 
> **Stability fixes in test flows and helpers.** Perps and Predict
performance specs adjust timer boundaries and UI sequencing (including
handling pre-existing positions), `Flows.selectAccountDevice` becomes
device-matrix driven and waits for account syncing, and
gesture/screen-object tweaks improve tap reliability, keyboard dismissal
(iOS+Android), and a few selectors/timeouts.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
1a4591a. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Author: javiergarciavera

appwright/fixtures/performance-test.js

@@ -1,11 +1,31 @@
 import { test as base } from 'appwright';
 import { PerformanceTracker } from '../reporters/PerformanceTracker.js';
 import QualityGatesValidator from '../utils/QualityGatesValidator.js';
+import {
+  markQualityGateFailure,
+  hasQualityGateFailure,
+  getTestId,
+} from '../utils/QualityGateError.js';
 
 // Create a custom test fixture that handles performance tracking and cleanup
 export const test = base.extend({
   // eslint-disable-next-line no-empty-pattern
   performanceTracker: async ({}, use, testInfo) => {
+    const testId = getTestId(testInfo);
+
+    // Skip retry if previous attempt failed due to quality gates
+    // Quality gate failures should NOT be retried - the measurement was valid, only threshold exceeded
+    if (testInfo.retry > 0 && hasQualityGateFailure(testId)) {
+      console.log(
+        `⏭️ Skipping retry for "${testInfo.title}" - previous attempt failed due to Quality Gates (threshold exceeded, not a test execution error)`,
+      );
+      testInfo.skip(
+        true,
+        'Skipped retry: Quality Gates failed in previous attempt. Performance threshold was exceeded but test execution was successful.',
+      );
+      return;
+    }
+
     const performanceTracker = new PerformanceTracker();
 
     // Provide the tracker to the test
@@ -39,11 +59,22 @@ export const test = base.extend({
     );
     if (hasThresholds) {
       console.log('🔍 Validating quality gates...');
-      QualityGatesValidator.assertThresholds(
-        testInfo.title,
-        performanceTracker.timers,
-      );
-      console.log('✅ Quality gates PASSED');
+      try {
+        QualityGatesValidator.assertThresholds(
+          testInfo.title,
+          performanceTracker.timers,
+        );
+        console.log('✅ Quality gates PASSED');
+      } catch (error) {
+        // Mark this test as failed due to quality gates so retries are skipped
+        if (error.isQualityGateError) {
+          markQualityGateFailure(testId);
+          console.log(
+            '🚫 Quality gates FAILED - retries will be skipped for this test',
+          );
+        }
+        throw error;
+      }
     }
 
     console.log('🔍 Looking for session ID...');

appwright/reporters/custom-reporter.js

@@ -2,6 +2,7 @@
 import { PerformanceTracker } from './PerformanceTracker';
 import { AppProfilingDataHandler } from './AppProfilingDataHandler';
 import QualityGatesValidator from '../utils/QualityGatesValidator';
+import { clearQualityGateFailures } from '../utils/QualityGateError.js';
 import fs from 'fs';
 import path from 'path';
 
@@ -15,6 +16,17 @@ class CustomReporter {
 
   // We'll skip the onStdOut and onStdErr methods since the list reporter will handle those
 
+  /**
+   * Called once before running tests.
+   * Clears quality gate failures from previous test runs.
+   */
+  onBegin() {
+    console.log(
+      '🚀 Test suite starting: Clearing quality gate failures from previous runs...',
+    );
+    clearQualityGateFailures();
+  }
+
   onTestEnd(test, result) {
     // Create a unique test identifier to avoid duplicate processing
     // Use test title and project name as unique ID

appwright/tests/performance/login/perps-position-management.spec.js

@@ -49,17 +49,13 @@ test.describe(PerformancePreps, () => {
     test.setTimeout(10 * 60 * 1000); // 10 minutes
 
     const selectPerpsMainScreenTimer = new TimerHelper(
-      'Select Perps Main Screen',
+      'Perps tutorial screen visible',
       { ios: 2000, android: 2000 },
       device,
     );
-    const skipTutorialTimer = new TimerHelper(
-      'Skip Tutorial',
-      { ios: 1600, android: 2500 },
-      device,
-    );
+
     const selectMarketTimer = new TimerHelper(
-      'Select Market BTC',
+      'Market list screen visible',
       { ios: 7500, android: 7500 },
       device,
     );
@@ -69,69 +65,69 @@ test.describe(PerformancePreps, () => {
       device,
     );
     const openPositionTimer = new TimerHelper(
-      'Open Long Position',
+      'Position opened',
       { ios: 10500, android: 20000 },
       device,
     );
-    const setLeverageTimer = new TimerHelper(
-      'Set Leverage',
-      { ios: 13500, android: 13500 },
-      device,
-    );
-    const closePositionTimer = new TimerHelper(
-      'Close Position',
-      { ios: 8500, android: 9500 },
+
+    const MarketDetailsScreenTimer = new TimerHelper(
+      'Market Details Screen',
+      { ios: 10000, android: 10000 },
       device,
     );
+
     await screensSetup(device);
     await login(device);
 
     // Perps requires independent account for each device to avoid clashes when running tests in parallel
     await selectAccountDevice(device, testInfo);
 
     await TabBarModal.tapActionButton();
+    await WalletActionModal.tapPerpsButton();
+    await selectPerpsMainScreenTimer.measure(async () => {
+      await PerpsTutorialScreen.isContainerDisplayed();
+    });
 
-    await selectPerpsMainScreenTimer.measure(() =>
-      WalletActionModal.tapPerpsButton(),
-    );
+    await PerpsTutorialScreen.tapSkip();
+    await selectMarketTimer.measure(async () => {
+      await PerpsMarketListView.isHeaderVisible();
+    });
 
-    // Skip tutorial
-    await skipTutorialTimer.measure(() => PerpsTutorialScreen.tapSkip());
+    await PerpsMarketListView.selectMarket('BTC');
 
-    // Selecting BTC market
-    await selectMarketTimer.measure(() =>
-      PerpsMarketListView.selectMarket('BTC'),
+    await MarketDetailsScreenTimer.measure(
+      async () => await PerpsPositionDetailsView.isContainerDisplayed(),
     );
-
-    // TODO: Add a check to see if the position is open
-    // If position open, fail the test
+    // Check if there's an existing position and close it before continuing
     if (await PerpsPositionDetailsView.isPositionOpen()) {
-      throw new Error('Position is already open');
+      console.log(
+        '⚠️ Position already open, closing it before continuing with the test...',
+      );
+      await PerpsPositionDetailsView.closePositionWithRetry();
+      console.log('✅ Existing position closed successfully');
     }
 
+    await PerpsMarketDetailsView.tapLongButton();
     // Open Position
-    await openOrderScreenTimer.measure(() =>
-      PerpsMarketDetailsView.tapLongButton(),
+    await openOrderScreenTimer.measure(async () =>
+      PerpsOrderView.checkOrderScreenVisible(),
     );
 
-    // Set leverage to 40x
-    await setLeverageTimer.measure(() => PerpsOrderView.setLeverage(40));
+    await PerpsOrderView.setLeverage(40);
+    await PerpsOrderView.tapPlaceOrder();
 
-    await openPositionTimer.measure(() => PerpsOrderView.tapPlaceOrder());
-
-    // Close Position
-    await closePositionTimer.measure(() =>
-      PerpsPositionDetailsView.closePositionWithRetry(),
+    await openPositionTimer.measure(
+      async () => await PerpsPositionDetailsView.isPositionOpen(),
     );
 
+    await PerpsPositionDetailsView.closePositionWithRetry();
+
     performanceTracker.addTimers(
       selectPerpsMainScreenTimer,
-      skipTutorialTimer,
       selectMarketTimer,
       openOrderScreenTimer,
-      setLeverageTimer,
       openPositionTimer,
-      closePositionTimer,
+      MarketDetailsScreenTimer,
     );
     await performanceTracker.attachToTest(testInfo);
   });

appwright/tests/performance/login/predict/predict-market-details.spec.js

@@ -38,16 +38,18 @@ test.describe(PerformancePredict, () => {
 
     // Login to the app
     await login(device);
+    console.log('Tap Action Button');
     await TabBarModal.tapActionButton();
+    console.log('Tapped Action Button');
 
     // Timer 2: Open predictions tab (threshold: 5000ms + 10% = 5500ms)
     const timer2 = new TimerHelper(
       'Time since user taps Predict button until Predict Market List is displayed',
       { ios: 2800, android: 4000 },
       device,
     );
+    await WalletActionModal.tapPredictButton();
     await timer2.measure(async () => {
-      await WalletActionModal.tapPredictButton();
       await PredictMarketListScreen.isContainerDisplayed();
     });
 
@@ -57,8 +59,8 @@ test.describe(PerformancePredict, () => {
       { ios: 17000, android: 13000 },
       device,
     );
+    await PredictMarketListScreen.tapMarketCard('trending', 2); // second card to avoid flakiness for a promoted card
     await timer3.measure(async () => {
-      await PredictMarketListScreen.tapMarketCard('trending', 1);
       await PredictDetailsScreen.isVisible();
     });
 
@@ -68,8 +70,8 @@ test.describe(PerformancePredict, () => {
       { ios: 7800, android: 7800 },
       device,
     );
+    await PredictDetailsScreen.tapAboutTab();
     await timer4.measure(async () => {
-      await PredictDetailsScreen.tapAboutTab();
       await PredictDetailsScreen.isAboutTabContentDisplayed();
       await PredictDetailsScreen.verifyVolumeTextDisplayed();
     });

appwright/utils/Flows.js

@@ -18,36 +18,59 @@ import RewardsGTMModal from '../../wdio/screen-objects/Modals/RewardsGTMModal.js
 import AppwrightGestures from '../../tests/framework/AppwrightGestures.js';
 import AppwrightSelectors from '../../tests/framework/AppwrightSelectors.js';
 import { expect } from 'appwright';
+import deviceMatrix from '../device-matrix.json' with { type: 'json' };
+
+/**
+ * Builds a device-to-account mapping from device-matrix.json
+ * Account assignments:
+ * - Account 1: Default (first device in each platform category with 'low' category)
+ * - Account 3: First Android device with 'high' category
+ * - Account 4: First iOS device with 'high' category
+ * - Account 5: Second iOS device (low category)
+ * - Account 2: Reserved for 'stable' testing (not used in this function)
+ */
+function buildDeviceAccountMapping() {
+  const mapping = {};
+
+  // Process Android devices
+  deviceMatrix.android_devices.forEach((device, index) => {
+    if (device.category === 'high') {
+      mapping[device.name] = 'Account 3';
+    } else if (device.category === 'low') {
+      // Low category Android devices use default Account 1
+      mapping[device.name] = null;
+    }
+  });
+
+  // Process iOS devices
+  deviceMatrix.ios_devices.forEach((device, index) => {
+    if (device.category === 'high') {
+      mapping[device.name] = 'Account 4';
+    } else if (device.category === 'low') {
+      mapping[device.name] = 'Account 5';
+    }
+  });
+
+  return mapping;
+}
+
+// Build the mapping once at module load
+const deviceAccountMapping = buildDeviceAccountMapping();
 
 export async function selectAccountDevice(device, testInfo) {
   // Access device name from testInfo.project.use.device
   const deviceName = testInfo.project.use.device.name;
   console.log(`📱 Device executing the test: ${deviceName}`);
 
-  let accountName;
-
-  // Define account mapping based on device name
-  // The device names must match those in appwright.config.ts or device-matrix.json
-  switch (deviceName) {
-    case 'Samsung Galaxy S23 Ultra':
-      accountName = 'Account 3';
-      break;
-    case 'Google Pixel 8 Pro':
-      console.log(
-        `🔄 Account 1 is selected by default in the app for device: ${deviceName}`,
-      );
-      return;
-    case 'iPhone 16 Pro Max':
-      accountName = 'Account 4';
-      break;
-    case 'iPhone 12':
-      accountName = 'Account 5';
-      break;
-    default:
-      console.log(
-        `🔄 Account 1 is selected by default in the app for device: ${deviceName}`,
-      );
-      return;
+  // Get account name from the dynamic mapping
+  const accountName = deviceAccountMapping[deviceName];
+
+  // If no account mapping exists or accountName is null, use default Account 1
+  if (!accountName) {
+    console.log(
+      `🔄 Account 1 is selected by default in the app for device: ${deviceName}`,
+    );
+    return;
   }
   // Account 2 is called stable and not used in this function
 
@@ -62,6 +85,7 @@ export async function selectAccountDevice(device, testInfo) {
   // Perform account switch
   await WalletMainScreen.tapIdenticon();
   await AccountListComponent.isComponentDisplayed();
+  await AccountListComponent.waitForSyncingToComplete();
   await AccountListComponent.tapOnAccountByName(accountName);
 
   // Verify we are back on main screen (tapping account usually closes modal)