Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exploration: Playwright GH action workflow #7922

Merged
merged 10 commits into from
Jan 12, 2024

Conversation

Jinksi
Copy link
Member

@Jinksi Jinksi commented Dec 15, 2023

Based on PR #7671

This PR is an initial attempt/exploration to get Playwright tests running in GH actions, consistent with the existing e2e workflow files.

Note this doesn't run Puppeteer spec files, only the few example Playwright tests from the e2e-pw dir.

Outputs the following Playwright HTML report as a GH action artefact (see this workflow run):

image

Testing instructions

  • Ensure GH action passes and runs all e2e tests: E2E Playwright Tests.

  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@Jinksi Jinksi marked this pull request as ready for review December 15, 2023 05:54
@Jinksi Jinksi requested a review from a team as a code owner December 15, 2023 05:54
@botwoo
Copy link
Collaborator

botwoo commented Dec 15, 2023

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 7922 or branch name try/playwright-e2e-tests-gh-actions in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: 18f7a96
  • Build time: 2024-01-11 01:34:30 UTC

Note: the build is updated when a new commit is pushed to this PR.

Copy link
Contributor

github-actions bot commented Dec 15, 2023

Size Change: 0 B

Total Size: 1.27 MB

ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.06 kB
release/woocommerce-payments/assets/css/success.css 158 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css 1.81 kB
release/woocommerce-payments/dist/blocks-checkout.css 1.81 kB
release/woocommerce-payments/dist/blocks-checkout.js 84.8 kB
release/woocommerce-payments/dist/checkout-rtl.css 318 B
release/woocommerce-payments/dist/checkout.css 319 B
release/woocommerce-payments/dist/checkout.js 36.8 kB
release/woocommerce-payments/dist/index-rtl.css 37.8 kB
release/woocommerce-payments/dist/index.css 37.9 kB
release/woocommerce-payments/dist/index.js 289 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.05 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 3.4 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js 60.5 kB
release/woocommerce-payments/dist/multi-currency.css 3.4 kB
release/woocommerce-payments/dist/multi-currency.js 55.7 kB
release/woocommerce-payments/dist/order-rtl.css 676 B
release/woocommerce-payments/dist/order.css 679 B
release/woocommerce-payments/dist/order.js 42.1 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css 1.31 kB
release/woocommerce-payments/dist/payment-gateways.css 1.31 kB
release/woocommerce-payments/dist/payment-gateways.js 39.3 kB
release/woocommerce-payments/dist/payment-request-rtl.css 153 B
release/woocommerce-payments/dist/payment-request.css 153 B
release/woocommerce-payments/dist/payment-request.js 12.9 kB
release/woocommerce-payments/dist/product-details.js 898 B
release/woocommerce-payments/dist/settings-rtl.css 10.4 kB
release/woocommerce-payments/dist/settings.css 10.4 kB
release/woocommerce-payments/dist/settings.js 233 kB
release/woocommerce-payments/dist/subscription-edit-page.js 669 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 519 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 519 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 20.4 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 693 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 117 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 117 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 19.5 kB
release/woocommerce-payments/dist/tos-rtl.css 230 B
release/woocommerce-payments/dist/tos.css 231 B
release/woocommerce-payments/dist/tos.js 22 kB
release/woocommerce-payments/dist/woopay-express-button-rtl.css 153 B
release/woocommerce-payments/dist/woopay-express-button.css 153 B
release/woocommerce-payments/dist/woopay-express-button.js 52.3 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.18 kB
release/woocommerce-payments/dist/woopay.css 4.19 kB
release/woocommerce-payments/dist/woopay.js 71.8 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 622 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 812 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.43 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.01 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 522 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 581 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/babel.config.js 160 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.css 2.37 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.js 13.5 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.rtl.css 2.37 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.03 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 291 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 403 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 299 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 742 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 572 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 411 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.8 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.83 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 502 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 355 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 429 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 781 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.27 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 392 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.05 kB

compressed-size-action

@Jinksi Jinksi marked this pull request as draft December 15, 2023 06:04
@Jinksi Jinksi changed the title Add Playwright GH actions workflow Add Playwright GH action workflow Dec 15, 2023
Copy link
Contributor

@jbordonado jbordonado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, is that just a PR for setting up the foundation to have a working workflow that we can rely upon to run Playwright tests?

I'm sure there are syntax differences on the way to write tests, so we can't just start running tests with this PR, right? Just checking, the PR description made me think that the PR would allow us to run the tests already

Anyway, thanks for exploring this 👍

path: ${{ env.WCPAY_ARTIFACT_DIRECTORY }}

- name: Setup E2E environment
uses: ./.github/actions/e2e-pw/env-setup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
uses: ./.github/actions/e2e-pw/env-setup
uses: ./.github/actions/e2e/env-setup

I think we can reuse the ./.github/actions/e2e/env-setup composite action, and add the specific PW step for installing it right after, instead of creating a new env-setup action from scratch.

- name: Install Playwright
          shell: bash
          run: |
              echo "::group::Install Playwright"
              npx playwright install
              echo "::endgroup::"

Or do you foresee any more specific setup to be done for PW, beside installing it (which should be done automatically when we add the playwright dependency in our package.json; why haven't we added playwright as dev dependency in that PR?)?

In any case, the e2e-pw composite action could run the ./.github/actions/e2e/env-setup and add specific steps after, if we really want to encapsulate the whole PW setup thing in a different file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why haven't we added playwright as dev dependency in that PR?

@playwright/test is installed as a dev dependency, but it seems that Playwright needs to run an additional browser installation step with npx playwright install (see docs).

"@playwright/test": "1.39.0",

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can reuse the ./.github/actions/e2e/env-setup composite action

I've made this change in 83c1e6b and added the specific Playwright install step.

@Jinksi
Copy link
Member Author

Jinksi commented Dec 15, 2023

To clarify, is that just a PR for setting up the foundation to have a working workflow that we can rely upon to run Playwright tests?

Sorry for the confusion!

This (and PR #7671) has only explored the foundation, running the few Playwright tests added to the e2e-pw directory and doesn't run any of our current Puppeteer e2e tests.

@Jinksi Jinksi changed the title Add Playwright GH action workflow Exploration: Playwright GH action workflow Dec 15, 2023
@Jinksi Jinksi marked this pull request as ready for review December 22, 2023 04:37
@Jinksi
Copy link
Member Author

Jinksi commented Jan 12, 2024

I'll merge this into #7671 to simplify the work-in-progress into one PR.

@Jinksi Jinksi merged commit 8313e50 into try/playwright-e2e-tests Jan 12, 2024
21 of 22 checks passed
@Jinksi Jinksi deleted the try/playwright-e2e-tests-gh-actions branch January 12, 2024 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants