W-13747172 - E2e a11y integration by alexvuong · Pull Request #2381 · SalesforceCommerceCloud/pwa-kit

alexvuong · 2025-04-30T00:28:42Z

Description

As more developers/external contribute to our monorepo, we want to ensure that no more a11y issues to our template.
This PR integrates a11y tests using snapshots to capture existing issues on each page of the template.

Why using snapshots?

Since our templates currently have some a11y that is not fixed due to attribute injection from Chakra. For example, search input a11y issue where we are using form as a button trigger. Chakra UI will inject form element with aria-expanded which is not allowed by a11y guideline. To resolve this issue, we may have to implement a more well-round solution than current implementation.

Therefore, we are using snapshot method to capture known issues without having to explicit excludes in our a11y tests.

What is the catch?

We have to commit snapshot files into our source code, and will need to update snapshots if we want to except new issues.
The tests will run against E2E env that runs together with all other E2E tests. Meaning if there is any new a11y, we won't know until the next nightly run.

What is used?

This PR uses playwright axe-core to integrate a11y tests. See more here.

Types of Changes

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Documentation update
Breaking change (could cause existing functionality to not work as expected)
Other changes (non-breaking changes that does not fit any of the above)

Breaking changes include:

Removing a public function or component or prop

Adding a required argument to a function

Changing the data type of a function parameter or return value

Adding a new peer dependency to package.json

Changes

(change1)

How to Test-Drive This PR

(step1)

Checklists

General

Changes are covered by test cases
CHANGELOG.md updated with a short description of changes (not required for documentation updates)

Accessibility Compliance

You must check off all items in one of the follow two lists:

There are no changes to UI

or...

Changes were tested with a Screen Reader (iOS VoiceOver or Android Talkback) and had no issues
Changes comply with WCAG 2.0 guidelines levels A and AA
Changes to common UI patterns and interactions comply with WAI-ARIA best practices

Localization

Changes include a UI text update in the Retail React App (which requires translation)

cc-prodsec · 2025-04-30T00:28:52Z

🎉 Snyk checks have passed. No issues have been found so far.

✅ security/snyk check is complete. No issues have been found. (View Details)

✅ license/snyk check is complete. No issues have been found. (View Details)

shethj · 2025-04-30T18:47:21Z

e2e/tests/desktop/__snapshots__/guest/cart-a11y-violations.json

@@ -0,0 +1,34 @@
+[


@alexvuong Did you manually type this out or is there a way to generate these snapshot files ? Can you add docs to the top of the file pointing to snapshot generation ?

yea to generate snapshots, we do npx run test --update-snapshots. I think it is a good idea to put that command in the npm script. Let me add that in

vcua-mobify

Approving. The change looks straightforward - we've generated snapshots to use as a baseline for our tests.

Is there a before & after of the tests without/with the snapshots just to compare?

shethj · 2025-05-03T01:14:04Z

package.json

    "start": "npm start --prefix packages/template-retail-react-app",
    "pretest": "npm run lint",
    "test:e2e": "npx playwright test",
+    "test:e2e:update-snapshots": "npx playwright test --update-snapshots",


Do we need to run this each time ?
If yes then should we be adding a step in the CI workflow to run this before the tests run ?

We don't need to run it each, only when we want to update snapshots

alexvuong · 2025-05-05T18:08:28Z

Approving. The change looks straightforward - we've generated snapshots to use as a baseline for our tests.

Is there a before & after of the tests without/with the snapshots just to compare?

There is none, the snapshots are to capture current state of our E2e Env. We don't compare before or after with snapshots. They are to prevent any new a11y coming up in the future. In the future, if there is a new a11y and we think it is okay to let it existing, we will come back and update snapshots to include the new issue.

…W-18895152@ (#2712) * Add template version to generator script for E2E tests * Update generator script to use templateVersion * Make templateVersion optional * make template version optional in validator script * Fix passing in template version as option * Don't pass in template version if undefined * Don't pass in templateVersion if undefined * Include full version string * Pin cc-datacloud version * Fix destructuring error in use-datacloud hook * Remove fallback to fix datacloud error * Bump versions for monorepo and packages * Update changelog * Update changelog * pin "@salesforce/cc-datacloud-typescript": "1.0.10" * Fix incorrect changelog version * Bump cc-datacloud-typescript version to 1.1.2 * Fix changelog * W-13747172 - E2e a11y integration (#2381) * set up a11y tests in e2e * @W-17466740 - Add Confirm New Password input field (#2395) * add confirm new password field * add tests * add translations * update changelog * lint * add translation file * revert eslint delete * fix account page tests * lint * add test case for error state * Fix a11y e2e (#2398) * fix a11y flow ---- Co-authored-by: Jainam Sheth <j.sheth@salesforce.com> * @W-18348131: Fix inconsistency between dwsid and access token when hybrid authentication is enabled. (#2397) * clear sfra access token before storing token response * Fix a11y e2e flaky tests (#2424) * fix a11y e2e flaky tests * [App Extensibility ⚙️] Fix Override Regression (@W-18485954@) (#2418) * Add missing fileExtensions & Use SRC_FOLDER to get project's path * Support overrides in generated project * Undo test config * Clean up * PR Feedback * More clean up removing SRC_FOLDER to get project's path * PR Feedback * PR Feedback * PR Feedback * Remove `template-retail-react-app` again, accidentally re-added * Initial removal of upgradeability, targets only `pwa-kit-extension-sdk` for deletion, cleanup of connected references to come later * Remove references to `pwa-kit-extension-sdk` in `package.json` files * Remove some missed `package.json` files * remove all `setup-app.ts` and `setup-server.ts` files, along with `pwa-kit-dev/src/ssr/server/test_fixtures/extensions/*` code * Remove upgradeability from `extension-chakra-store-locator` * Remove upgradeability: `useExtensionConfig` => `useConfig()` / `utils/get-extension-config` => `utils/get-config` / `ApplicationExtensionConfig` => `Config` * Prune references to: `ApplicationExtensionsProvider`, `setup-app`, `buildBabelExtensibilityArgs`, `getConfiguredExtensions`, `validateExtensionDependencies`, `validateAppConfiguration`, `applyApplicationExtensions`, `useApplicationExtensions` * Remove `overridable` directive / strings * remove more `setup-server` bits * fix bad merge * `extension-chakra-storefront` => `template-chakra-storefront` * `extension-chakra-storefront` => `template-chakra-storefront` * Revert "`extension-chakra-storefront` => `template-chakra-storefront`" This reverts commit 03675db. * `extension-chakra-storefront` => `template-chakra-storefront` * more cleanup, fixing plumbing / wiring in internals * More wire up / cleanup migrating extensibility * WIP (broken) mostly working `_app/index` and `_app-config/index` but loading `appConfig` (now derived from `default.json` instead of `constants.js`) is causing an infinite re-render loop * WIP (still broken) infinite render re-loop, seems some props passed to a provider are unstable between SSR and client-side pass * Working now (server side pass), fixed infinite loop, root cause ended up being lack of `mobify.proxyConfigs` in `default.json`, plus bad prop drilling in `ssr.js` * `import {useConfig} from '../hooks'` => `import {getConfig} from '@salesforce/pwa-kit-runtime/utils/ssr-config'` * Add gitignore (basic for now) * cleanup * cleanup * Cleanup upgradeability extensions code, `pages` / `pages.Home.path` was a config used by `setup-app.ts` * fix `getConfig` circular reference * Fix legacy references to page-based config related to extensibility * More cleanup * Fix references to missing configuration, add missing to `default.json` change casing * Restructure: `/src/assets` => `/assets` * Fix inconsistent `withCommerceSdkReact` hook. We are using V3 code in `components/_app/index` and the `with-commerce-sdk-react` hook was on the V4 (extensions) version * Fix `config.homeHref` => `config.pages.home.path` * cleanup * tree shake and integration with create-mobify-app * fixed typo * tree shake tests * removed unused import * Move `src/constants.js` => `config/constants.js` * re-structure folders `/src/*` => `/**` * Cleanup (remove `above-header` template hook) remove unused `with-layout.tsx` * checksum * dependencies * removed command line code * added missing export * update lockfile * fixed plugins length check * Add back missing translation compile scripts + remove V3 extensibility * Cleaning up V3 `ccExtensibility` + V4 extensions * Remove unused scripts * Remove more upgradeability code * more upgradeability removal * Add default chakra-storefront answers * back out debugging code * Introduce fallback value pattern and differentiation between `interactive` an `non-interactive` versions of template presets * default to non-interactive mode * Add to CHANGELOG * make typescript template --preset interactive by default * lint fix * fixed linting errors * lint fix * more lint fixes * remove unused HOCs * more unused legacy upgradeability cleanup * remove test for `cc_overrides` * delete upgradeability test cases * Rollback `useServerContext` hook added to `commerce-sdk-react` for upgradeability * Revert "Rollback `useServerContext` hook added to `commerce-sdk-react` for upgradeability" This reverts commit 4975fef. * attempting to fix `commerce-sdk-react` test failures * Refactor store locator to no longer use `useExtensionStore` hook * fix broken test in `extension-chakra-store-locator` * bump threshold down to pass tests because this will all fold into the template in V4 anyway * this is not actually mocking `getConfig`, it was mocking `get-extension-config` and got renamed at commit b5d5a9d when it should have been deleted * remove upgradeability code changes failing CI * attempt to fix unmocked file read * fix lint error pwa-kit-dev jest config * lint fix jest config * fix outdated references to `@salesforce/retail-react-app` * fixed renderWithProviders import * mock file read * added error handling * fixed for path mismatch in Windows * fix for windows path * fixed linter * more fix for Windows * fixed mockConfig for stores-list test * test fixes * lint * fixed iport path for locale jsons * fixed url test * fixed breadcrumb test * fixed store locator modal test * mock config fixes * more mock config fixes * mock config fixes * fixed mock config once and for all * fixed remaining tests * updated coverage collection list * renamed tree-shake to trim-extension * fixed linter * rename tree-shake.js in package.json * renamed retail-react-app to chakra-storefront and adjusted configs accordingly * keep preset ids as retail-react-app * revert the project name back to retail-react-app * moved source folders to under src/ and updated all references * fixed lint error * updated jest config * more mock data path fixes * @W-18670301 set default context preset, error handling in trim-extensions * move `app/components/_index` and other files in `app` dir to `src` dir * Remove references to outdated `template-retail-react-app` * update dir structure for non template-chakra-storefront` templates * fix bad pwa-kit-dev reference to `app/main` and `app/ssr` (now `src/main` / `src/ssr`) * @W-18670301 - added prettier and tuned tests * fixed lint errors * straggler rename `app/...` => `src/...` * change log * Add to CHANGELOGs * add missing CHANGELOG * Fix inconsistencies: `app/static` => `src/static` and `/static` => `src/static` * Move `app/static` => `src/static` for `pwa-kit-dev` and `test-commerce-sdk-react` * don't run tests on temporarily broken `extension-chakra-storefront` * fix path typo * fixed windows path * pull latest label from PR for skip changelog action * updated name for get_pr * fix translation pathing and associated failing tests * [feature] Rename V4 `app` dir to => `src` @W-18895152@ (#2693) * move `app/components/_index` and other files in `app` dir to `src` dir * Remove references to outdated `template-retail-react-app` * update dir structure for non template-chakra-storefront` templates * fix bad pwa-kit-dev reference to `app/main` and `app/ssr` (now `src/main` / `src/ssr`) * straggler rename `app/...` => `src/...` * Add to CHANGELOGs * add missing CHANGELOG * Fix inconsistencies: `app/static` => `src/static` and `/static` => `src/static` * Move `app/static` => `src/static` for `pwa-kit-dev` and `test-commerce-sdk-react` * don't run tests on temporarily broken `extension-chakra-storefront` * fix path typo * fix translation pathing and associated failing tests * Refactor `pwa-kit-create-app` handlebars template files to new dir structure * drop `@salesforce` from template namespace since we're not publishing to NPM, update var name to reflect * Remove remaining references to `@salesforce/chakra-storefront` * fix naming to remove `@salesforce/chakra-storefront` * remove `@salesforce` folder name * Revert "remove `@salesforce` folder name" This reverts commit 34877b1. * remove `@salesforce` move files instead of rename * Migrate a bunch of `retail-react-app` strings to `chakra-storefront` * delete *-ext and `*-no-ext` CI runs and generator variants because we no longer have extensibility * * Fix bad merge conflict, force everything to `4.0.0-extensibility-preview.5` * Fix `routes.tsx` and `routes.tsx.hbs` * fix failing test --------- Co-authored-by: Jainam Sheth <j.sheth@salesforce.com> Co-authored-by: Jainam Sheth <99490559+shethj@users.noreply.github.com> Co-authored-by: Alex Vuong <alex.vuong@salesforce.com> Co-authored-by: Yuna Kim <84923642+yunakim714@users.noreply.github.com> Co-authored-by: Ujwala <59447810+unandyala@users.noreply.github.com> Co-authored-by: Adam Raya <adamraya@users.noreply.github.com> Co-authored-by: kzheng <kzheng@salesforce.com> Co-authored-by: Ken Zheng <33469941+kzheng-sfdc@users.noreply.github.com>

alexvuong added 5 commits April 25, 2025 14:44

set up a11y in e2e

dd01026

Merge branch 'develop' into e2e-a11y-integration

a3fd264

set up a11y in e2e

d10850b

set up a11y in e2e

4375945

set up a11y in e2e

1aad4b2

alexvuong requested a review from a team as a code owner April 30, 2025 00:28

shethj reviewed Apr 30, 2025

View reviewed changes

shethj previously approved these changes Apr 30, 2025

View reviewed changes

add command for update snapshots

c8be15d

alexvuong dismissed shethj’s stale review via c8be15d April 30, 2025 19:17

alexvuong added 5 commits April 30, 2025 12:40

Merge branch 'develop' into e2e-a11y-integration

55f8c13

update snapshots

ccf0a8c

update snapshots

48166ea

Merge branch 'develop' into e2e-a11y-integration

20f3fa8

update lock file

04e448d

vcua-mobify approved these changes May 3, 2025

View reviewed changes

shethj reviewed May 3, 2025

View reviewed changes

shethj approved these changes May 3, 2025

View reviewed changes

alexvuong merged commit 77732f2 into develop May 5, 2025
35 checks passed

alexvuong deleted the e2e-a11y-integration branch July 16, 2025 22:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

W-13747172 - E2e a11y integration#2381

W-13747172 - E2e a11y integration#2381
alexvuong merged 11 commits intodevelopfrom
e2e-a11y-integration

alexvuong commented Apr 30, 2025

Uh oh!

cc-prodsec commented Apr 30, 2025 •

edited

Loading

Uh oh!

shethj Apr 30, 2025 •

edited

Loading

Uh oh!

alexvuong Apr 30, 2025

Uh oh!

vcua-mobify left a comment

Uh oh!

shethj May 3, 2025

Uh oh!

alexvuong May 5, 2025

Uh oh!

alexvuong commented May 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

alexvuong commented Apr 30, 2025

Description

Why using snapshots?

What is the catch?

What is used?

Types of Changes

Changes

How to Test-Drive This PR

Checklists

General

Accessibility Compliance

Localization

Uh oh!

cc-prodsec commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎉 Snyk checks have passed. No issues have been found so far.

Uh oh!

shethj Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexvuong Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

vcua-mobify left a comment

Choose a reason for hiding this comment

Uh oh!

shethj May 3, 2025

Choose a reason for hiding this comment

Uh oh!

alexvuong May 5, 2025

Choose a reason for hiding this comment

Uh oh!

alexvuong commented May 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cc-prodsec commented Apr 30, 2025 •

edited

Loading

shethj Apr 30, 2025 •

edited

Loading