[UI Tests] - Add logout at launch capability #20891
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
| App Name |  WordPress Alpha |
|
| Configuration | Release-Alpha | |
| Build Number | pr20891-10992b6 | |
| Version | 22.6 | |
| Bundle ID | org.wordpress.alpha | |
| Commit | 10992b6 | |
| App Center Build | WPiOS - One-Offs #6011 |
Contributor
|
| App Name |  Jetpack Alpha |
|
| Configuration | Release-Alpha | |
| Build Number | pr20891-10992b6 | |
| Version | 22.6 | |
| Bundle ID | com.jetpack.alpha | |
| Commit | 10992b6 | |
| App Center Build | jetpack-installable-builds #5037 |
jostnes
changed the title
jostnes
commented
Jun 16, 2023
|
|
||
| XCTAssert(MySiteScreen.isLoaded()) | ||
| .verifyMySiteScreenLoaded() | ||
| .removeSelfHostedSite() |
Contributor
Author
There was a problem hiding this comment.
logOutDefaultWordPressComAccount works for WordPress accounts; needed to add this additional step for self-hosted sites for this work.
| // Unified email login/out | ||
| func testWPcomLoginLogout() throws { | ||
| let prologueScreen = try PrologueScreen().selectContinue() | ||
| try PrologueScreen().selectContinue() |
Contributor
Author
There was a problem hiding this comment.
this change is not really necessary for this change but i decided to take advantage to standardize this since there was 1 test in this file that had to be updated to work with this new solution
| */ | ||
| func testEmailMagicLinkLogin() throws { | ||
| let welcomeScreen = try WelcomeScreen().selectLogin() | ||
| try WelcomeScreen().selectLogin() |
Contributor
Author
There was a problem hiding this comment.
same as the above test
| let prologueScreen = try PrologueScreen() | ||
|
|
||
| try prologueScreen | ||
| try PrologueScreen() |
Contributor
Author
There was a problem hiding this comment.
same as the above test
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Currently, there are a lot of "Failed to launch" errors being logged in Buildkite logs during UI test runs. This was a side effect of deleting the app after each test run.
In this PR, I've removed
removeApp()fromtearDown()and replaced it with a new launch argument-logout-at-launchthat will be called at the start of each test. This will reset the state to log out at the start of every test run.This change also comes with bonus speed improvements as the app is no longer removed after each test run (previously, the removal happened on the UI level). Below are screenshots to better visualize the before and after of the change:
Before change
(using the current latest trunk build for this comparison)
From
.xresults:From Buildkite (build):
iPhone total duration: 16m 24s
iPad total duration: 21m 18s
After change
From
.xresults:From Buildkite (build):
iPhone total duration: 13m 16s (⬇️ 3m 8s)
iPad total duration: 18m 17s (⬇️ 3m 1s)
Testing
Errors like
[22:34:12]: ▸ 2023-06-15 22:34:12.514 xcodebuild[1865:17594] iOSSimulator: 5C8F2C2C-3188-492D-B8B5-AF71DED82192: Failed to launch app with identifier: com.automattic.jetpack and options:should no longer be logged in Buildkite logs and the test results file. CI should be 🟢