Skip to content

Commit d876391

Browse files
authored
ci: adapt automated tests metrics logic (#27552)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until the template has been completely filled out, and PR status checks have passed at least once. --> ## **Description** Updates QA stats collection to emit richer, more explicit metrics. Example: ``` ✅ QA stats written to ./qa-stats.json: { unit: { total_tests_run: 42997, predict_tests_run: 3604, selectors_tests_run: 1141, core_tests_run: 5153, other_tests_run: 4923, store_tests_run: 1047, tokens_tests_run: 240, bridge_tests_run: 1796, earn_tests_run: 1519, rewards_tests_run: 2037, util_tests_run: 2328, perps_tests_run: 5439, networksmanagement_tests_run: 234, reducers_tests_run: 641, card_tests_run: 2738, homepage_tests_run: 284, controllers_tests_run: 1440, confirmations_tests_run: 2502, components_base_tests_run: 323, components_hooks_tests_run: 784, settings_tests_run: 229, browsertab_tests_run: 232, multichainaccounts_tests_run: 356, ramp_tests_run: 2128, stake_tests_run: 207, component_library_tests_run: 1006, trending_tests_run: 325, total_tests_defined: 40267, total_tests_skipped: 17, line_coverage: 87.4, statement_coverage: 87, branch_coverage: 79.1, function_coverage: 83.3 }, component_view: { total_tests_run: 104, perps_tests_run: 32, earn_tests_run: 28, confirmations_tests_run: 5, predict_tests_run: 17, wallet_tests_run: 5, assetdetails_tests_run: 1, bridge_tests_run: 9, trendingview_tests_run: 6, walletactions_tests_run: 1, total_tests_defined: 102, total_tests_skipped: 0, line_coverage: 15.8, statement_coverage: 15.7, branch_coverage: 9.4, function_coverage: 13 }, e2e: { main_tests_run: 140, main_android_tests_run: 140, main_ios_tests_run: 140, flask_tests_run: 72, flask_android_tests_run: 72, flask_ios_tests_run: 72, total_tests_run: 212, multichain_api_tests_run: 12, prediction_market_tests_run: 10, wallet_platform_tests_run: 35, network_expansion_tests_run: 10, confirmations_tests_run: 31, trade_tests_run: 12, identity_tests_run: 6, network_abstraction_tests_run: 12, accounts_tests_run: 2, ramps_tests_run: 3, card_tests_run: 6, perps_tests_run: 1, total_tests_defined: 225, total_tests_skipped: 18 }, performance: { total_tests_defined: 22, total_tests_skipped: 1, login_tests_defined: 11, mm_connect_tests_defined: 6, onboarding_tests_defined: 5 } } ``` <!-- Write a short description of the changes included in this pull request, also include relevant motivation and context. Have in mind the following questions: 1. What is the reason for the change? 2. What is the improvement/solution? --> ## **Changelog** <!-- If this PR is not End-User-Facing and should not show up in the CHANGELOG, you can choose to either: 1. Write `CHANGELOG entry: null` 2. Label with `no-changelog` If this PR is End-User-Facing, please write a short User-Facing description in the past tense like: `CHANGELOG entry: Added a new tab for users to see their NFTs` `CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker` (This helps the Release Engineer do their job more quickly and accurately) --> CHANGELOG entry: ## **Related issues** Fixes: ## **Manual testing steps** ```gherkin Feature: my feature name Scenario: user [verb for user action] Given [describe expected initial app state] When user [verb for user action] Then [describe expected outcome] ``` ## **Screenshots/Recordings** <!-- If applicable, add screenshots and/or recordings to visualize the before and after of your change. --> ### **Before** <!-- [screenshots/recordings] --> ### **After** <!-- [screenshots/recordings] --> ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I've included tests if applicable - [x] I've documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I've applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [x] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [x] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Moderate risk because it changes how QA metrics are keyed/collected and updates CI artifact production/retention; mismatches in artifact names or parsing could silently drop metrics or skew dashboards. > > **Overview** > Updates the QA stats pipeline to generate **richer, explicitly named metrics** (e.g., `*_tests_run`, `total_tests_defined`, `total_tests_skipped`) and adds static source scanning to estimate defined/skipped tests across unit, component-view, E2E smoke, and performance suites. > > The collector no longer depends on a passed `WORKFLOW_RUN_ID`; it now resolves the **latest successful `ci.yml` run on `main`** via the GitHub API and pulls additional artifacts to include **coverage percentages** for unit and component-view tests. > > CI workflows are adjusted to support this by producing/uploading new coverage-summary artifacts and setting **7-day retention** on several uploaded artifacts, and the `qa-stats.yml` workflow switches from `workflow_run` triggering to a **daily scheduled run**. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit ea63d48. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent 1db9700 commit d876391

7 files changed

Lines changed: 292 additions & 45 deletions

0 commit comments

Comments
 (0)