[UI Test] Enhance EditorPostSettings Screen Object and reduce testCreateScheduledPost() flakiness.

[tiagomar]

Description

testCreateScheduledPost() is currently the least reliable ui test in iPad with ~85% at the time of writing. While working on another tests, testCreateScheduledPost() started consistently failing locally for me, so I could take a closer look. The error I've seen doesn't match all the errors in CI but fixing it is a good improvement at least.

The test fails when trying to select the day in the Calendar. The element being tapped is staticText["1"] which was not hittable in my case. Checking the hierarchy, staticText["1"] in under a button that is hittable. This PR changes the firstCalendarDay locator from staticText["1"] to a button containing staticText["1"].

Testing

Run the test locally on iPhone and iPad.
CI must be 🟢.

[wpmobilebot]

📲 You can test the changes from this Pull Request in Jetpack Alpha by scanning the QR code below to install the corresponding build.

	App Name	Jetpack Alpha
	Configuration	Release-Alpha
	Build Number	pr21402-09fc217
	Version	23.1
	Bundle ID	com.jetpack.alpha
	Commit	09fc217
	App Center Build	jetpack-installable-builds #5868

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[wpmobilebot]

📲 You can test the changes from this Pull Request in WordPress Alpha by scanning the QR code below to install the corresponding build.

	App Name	WordPress Alpha
	Configuration	Release-Alpha
	Build Number	pr21402-09fc217
	Version	23.1
	Bundle ID	org.wordpress.alpha
	Commit	09fc217
	App Center Build	WPiOS - One-Offs #6827

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[jostnes]

Thanks for making the element stricter @tiagomar! Though I agree that it would probably not solve the flakiness entirely. I had a look at the test flakiness previously and have yet to find a proper solution, but the reason it's flaky, I believe, is due to the timing of the test.

The test expects to tap on the View button on the notice, seen here:

But when the test reaches the code to do the actual tap, the notice has disappeared and, in the process, will fail the test:

It's always tricky for UI tests to depend on unreliable elements like the notice, especially in CI runs.

[jostnes]

@tiagomar I've been thinking about the testCreateScheduledPost() test today, what do you think about removing the assert from viewPublishedPost(), this part:

WordPress-iOS/WordPress/UITestsFoundation/Screens/Editor/EditorNoticeComponent.swift

Lines 30 to 37 in 605427c

	// The publish notice has a joined accessibility label equal to: title + message
	// (the postTitle). It does not seem possible to target the specific postTitle label
	// only because of this.
	XCTAssertEqual(
	noticeViewTitle.label,
	String(format: "%@. %@", expectedNoticeTitle, postTitle),
	"Post title not visible on published post notice"
	)

And only depend on the asserts in verifyEpilogueDisplays(). I think because it takes time to do the assert and the eventual tap, if we remove the assert, the tap might have a higher chance of working while the notice is still displayed. WDYT? Would that help with the flakiness?

[tiagomar]

Good point, @jostnes! I checked the failures and 12 out of 17 apparently fail just like you described.

The assertion itself takes less than 1s in my machine, but it could take longer in CI. But in this case, I think that having a dedicated Screen Object to the notice is an overkill and the init can take maybe a couple seconds?

Another thing I noticed is that we wait up to 3 seconds to dismiss Blogging Reminder but it apparently doesn't show up for scheduled posts?

WordPress-iOS/WordPress/UITestsFoundation/Screens/Editor/BlockEditorScreen.swift

Lines 405 to 412 in f697472

	private func confirmPost(button: XCUIElement) throws {
	if FancyAlertComponent.isLoaded() {
	try FancyAlertComponent().acceptAlert()
	} else {
	button.tap()
	dismissBloggingRemindersAlertIfNeeded()
	}
	}

I'm working on a PR touching these points and you should be pinged soon. =]