Plans on JP: Update UI test to reach purchase result screen

[staskus]

Fixes #20790

Implementation

1. Use WireMock to mock web view pages instead of API endpoints
2. Create fake checkout and plans pages to be used in UI tests
3. Redirect to expected pages once the links are tapped
4. Use Environment to inject WireMock URL and use it instead of WP.com URL
5. Covered the whole flow with tests

To test:

• Confirm UI tests succeeding in CI

Regression Notes

1. Potential unintended areas of impact

Affecting free to paid plans flow

2. What I did to test those areas of impact (or what existing automated tests I relied on)

Manually retesting the whole purchase flow

3. What automated tests I added (or what prevented me from doing so)

Covered the whole free to paid plans flow with automated UI Tests

PR submission checklist:

• I have completed the Regression Notes.
• I have considered adding unit tests for my changes.
• I have considered adding accessibility improvements for my changes.
• I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

Video

Full.Flow.UI.Test.mp4

[staskus]

@guarani if you're interested, it looks like WireMock can also be used to mock web view content. Check the video from [00:17-] to see how it would look. We would essentially verify that our native code responds to the redirections as expected. This way we can simulate the whole flow. WDYT?

[wpmobilebot]

📲 You can test the changes from this Pull Request in WordPress Alpha by scanning the QR code below to install the corresponding build.

	App Name	WordPress Alpha
	Configuration	Release-Alpha
	Build Number	pr20805-51fe2ef
	Version	23.5
	Bundle ID	org.wordpress.alpha
	Commit	51fe2ef
	App Center Build	WPiOS - One-Offs #7523

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[wpmobilebot]

📲 You can test the changes from this Pull Request in Jetpack Alpha by scanning the QR code below to install the corresponding build.

	App Name	Jetpack Alpha
	Configuration	Release-Alpha
	Build Number	pr20805-51fe2ef
	Version	23.5
	Bundle ID	com.jetpack.alpha
	Commit	51fe2ef
	App Center Build	jetpack-installable-builds #6550

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[guarani]

@guarani if you're interested, it looks like WireMock can also be used to mock web view content. Check the video from [00:17-] to see how it would look. We would essentially verify that our native code responds to the redirections as expected. This way we can simulate the whole flow. WDYT?

This seems like an interesting approach. I watched the screen recording and agree this test can ensure that the app can handle the redirects.

On Calypso a test for Automattic/wp-calypso#77029 could verify the sequence of screens and redirect URLs don't get changed.

[staskus]

This seems like an interesting approach. I watched the screen recording and agree this test can ensure that the app can handle the redirects.

Yes. These are not E2E tests, only native UI tests. Just like with mocked APIs, we expect our app to behave in a certain way given the input.

On Calypso a test for Automattic/wp-calypso#77029 could verify the sequence of screens and redirect URLs don't get changed.

👍 Good idea

[staskus]

@guarani

Ready for review. The whole free to paid plans flow on the native side is now covered with UI tests

[guarani]

👋 While the changes look good, I keep running into the issue where the dashboard card doesn't appear and the UI test keeps trying to scroll the screen up to find it (similar to #20728 (review)). It also works the second time I run the app.

I re-read your comment, #20728 (comment), and the fix you added (reloading local cards after the app fetches the site domains), but don't see anything wrong.

Stopping at breakpoints seems to affect whether the card shows up or not, so I printed hasLoadedDomains in both the failing scenario (1st run after fresh install) and the passing scenario (2nd run) and there's a difference:

Not working	Working

I thought to check if this is reproducible on trunk and it isn't. I did two fresh installs and both worked the first time. Have you tried a fresh install on this branch? I'm curious if you can reproduce.

[staskus]

I thought to check if this is reproducible on trunk and it isn't. I did two fresh installs and both worked the first time. Have you tried a fresh install on this branch? I'm curious if you can reproduce.

It is reproducible but only sometimes. In a morning investigation, I couldn't point out exactly why it would happen. Even in your "not working" scenario, the third reload of the free to-paid plan card indicates that domains are loaded so the card should appear. I continue investigating.

[staskus]

I found the issue that hasMappedDomain unexpectedly returns true with these internal values, causing shouldShowCard to return false.

• API-Mocks have rest_v11_sites_106707880.json that matches urlPath": "/rest/v1.1/sites/106707880 and sets different values for "URL" and "unmapped_url" fields.
• rest_v11_sites_106707880.json matches urlPathPattern": "/rest/v1.(1|2)/me/sites that shares the same siteID but sets the same "URL" and "unmapped_url" fields.
• rest_v11_sites_106707880_settings.json that shares the same siteID but has a different "URL" field from the previous one

I haven't identified how these API mocks are matched and how it differs between the first or second launches. However, after making the mocks consistent, I could no longer reproduce the issue in multiple attempts.

Could you try it out again, @guarani, thanks!

[guarani]

👋 Hey @staskus, I gave this a try today and weirdly still getting the same issue. I'd love to investigate more today, but I want to prioritise manually testing the flows since the project is close to finishing.

Do you agree with de-prioritizing this until we're confident we've done enough manual testing? I appreciate you investigating this and I think once it's working, having the UI test here for the happy path will be very valuable to catch regressions going forward.

[staskus]

Do you agree with de-prioritizing this until we're confident we've done enough manual testing?

Yes, we can wait given the issues we're facing in producing these tests.