Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix E2E shopper tests around 3DS and UPE settings #8123

Merged
merged 9 commits into from
Feb 5, 2024
Merged

Conversation

bborman22
Copy link
Contributor

@bborman22 bborman22 commented Feb 2, 2024

This PR should fix the E2E shopper tests on pull requests.

Changes proposed in this Pull Request

There were two particular failures that were occurring in previous runs:

Saved cards  › when using a 3DS card added on checkout › should save the card

    : Timeout - Async callback was not invoked within the 100000 ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 100000 ms timeout specified by jest.setTimeout.Error:
Saved cards  › when using a basic card added on checkout › should delete the card

    No node found for selector: #username
Enabled UPE with deferred intent creation › Enabled UPE with deferred intent creation › should successfully place order with Giropay

    TypeError: text is not iterable

The solution's I'm applying in this PR for the 3DS issue is to address the fact that the additional 'iframe[name="acsFrame"]' is no longer present with the retirement of 3DS1. I hadn't seen any particular announcement around this change from Stripe, but it is not unexpected as our current conditional for looking for that iframe states it was a 3DS1 specific feature. I haven't been able to determine when / how this was introduced and could have been just a change from Stripe, but would be nice to narrow this down.

The solution to the missing #username field on the saved cards tests was actually caused by a previous tests issues. In the shopper-checkout-purchase-with-upe-methods.spec the merchantWCP.disablePaymentMethod function was not completing, causing the merchant to not be logged out, causing the next test to fail when trying to log in. This was due to the use of Xpath when the function was expecting a selector. There was already some prior art in merchantWCP.enablePaymentMethod which I applied here as well. This was introduced in #7954 when converting selectors into Xpath.

The solution to the iterable issue was to add a postcode to the newly added upe-customers data. This was introduced in #8082 but was likely unnoticed as the E2E tests were failing for a while by this point.

One thing I considered was whether we should look to make those tests more resilient by first confirming we're logged out. I leaned away from that just because it seemed that it may cover up issues in other tests not getting "cleaned up" properly, but wanted to open that up for some discussion (and perhaps open an issue to handle in a seperate PR?).

Testing instructions

  • Run tests locally to confirm all tests in the shopper branch are passing: E2E_GROUP='wcpay' E2E_BRANCH='shopper' npm run test:e2e
  • Confirm that all GH checks are passing for [WC - latest | wcpay - shopper]

  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@botwoo
Copy link
Collaborator

botwoo commented Feb 2, 2024

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 8123 or branch name fix/e2e-tests in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: ceb78d5
  • Build time: 2024-02-05 16:45:04 UTC

Note: the build is updated when a new commit is pushed to this PR.

Copy link
Contributor

github-actions bot commented Feb 2, 2024

Size Change: 0 B

Total Size: 1.27 MB

ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.06 kB
release/woocommerce-payments/assets/css/success.css 158 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css 1.81 kB
release/woocommerce-payments/dist/blocks-checkout.css 1.81 kB
release/woocommerce-payments/dist/blocks-checkout.js 83.7 kB
release/woocommerce-payments/dist/checkout-rtl.css 318 B
release/woocommerce-payments/dist/checkout.css 319 B
release/woocommerce-payments/dist/checkout.js 35.9 kB
release/woocommerce-payments/dist/index-rtl.css 37 kB
release/woocommerce-payments/dist/index.css 37 kB
release/woocommerce-payments/dist/index.js 291 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.05 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 3.4 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js 61.2 kB
release/woocommerce-payments/dist/multi-currency.css 3.4 kB
release/woocommerce-payments/dist/multi-currency.js 56.6 kB
release/woocommerce-payments/dist/order-rtl.css 676 B
release/woocommerce-payments/dist/order.css 679 B
release/woocommerce-payments/dist/order.js 42.6 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css 1.31 kB
release/woocommerce-payments/dist/payment-gateways.css 1.31 kB
release/woocommerce-payments/dist/payment-gateways.js 40.2 kB
release/woocommerce-payments/dist/payment-request-rtl.css 153 B
release/woocommerce-payments/dist/payment-request.css 153 B
release/woocommerce-payments/dist/payment-request.js 12.2 kB
release/woocommerce-payments/dist/product-details.js 919 B
release/woocommerce-payments/dist/settings-rtl.css 10.3 kB
release/woocommerce-payments/dist/settings.css 10.3 kB
release/woocommerce-payments/dist/settings.js 235 kB
release/woocommerce-payments/dist/subscription-edit-page.js 669 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 519 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 519 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 19.4 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 710 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 117 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 117 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 18.4 kB
release/woocommerce-payments/dist/tos-rtl.css 230 B
release/woocommerce-payments/dist/tos.css 231 B
release/woocommerce-payments/dist/tos.js 20.9 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js 2.06 kB
release/woocommerce-payments/dist/woopay-express-button-rtl.css 153 B
release/woocommerce-payments/dist/woopay-express-button.css 153 B
release/woocommerce-payments/dist/woopay-express-button.js 51.3 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.18 kB
release/woocommerce-payments/dist/woopay.css 4.19 kB
release/woocommerce-payments/dist/woopay.js 70.9 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 622 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 812 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.43 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.01 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 522 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 581 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/babel.config.js 160 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.css 2.37 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.js 13.5 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.rtl.css 2.37 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.03 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 291 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 403 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 299 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 742 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 572 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 411 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.8 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.83 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 502 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 355 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 429 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 781 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.27 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 392 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.05 kB

compressed-size-action

@bborman22 bborman22 requested review from a team and lovo-h and removed request for a team February 2, 2024 17:19
@achyuthajoy achyuthajoy self-requested a review February 4, 2024 16:14
Copy link
Contributor

@achyuthajoy achyuthajoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the full E2E test suite against this branch(https://github.com/Automattic/woocommerce-payments/actions/runs/7775675801) and can confirm that the changes are working as expected and wcpay shopper tests are passing except for nightly tests.

Noticed multi currency test failures on nightly, but not related to this PR.

Thanks for working on this @bborman22. LGTM 🚢

@@ -197,7 +197,7 @@ export async function confirmCardAuthentication(
);
let challengeFrame = await challengeFrameHandle.contentFrame();
// 3DS 1 cards have another iframe enclosing the authorize form
if ( cardType.toUpperCase() === '3DS' ) {
if ( cardType.toUpperCase() === '3DS1' ) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bborman22 I haven't checked Stripe test cards, if there are any specific ones which are triggering 3DS1 check, but AFAIK there is no code which would trigger 3DS1 check here.

For the context: So this might a dead code now. I did so in #8121 to trick linter, otherwise it would complain on unused cardType argument :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, I tested all the cards we have defined and none triggered the 3DS1 flow, so I removed card type for this function entirely as it doesn't seem we need to look for 3DS1 any more.

@vbelolapotkov vbelolapotkov mentioned this pull request Feb 5, 2024
6 tasks
@bborman22 bborman22 added this pull request to the merge queue Feb 5, 2024
Merged via the queue into develop with commit 02194d1 Feb 5, 2024
22 of 26 checks passed
@bborman22 bborman22 deleted the fix/e2e-tests branch February 5, 2024 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants