Stabilize US bank account Lite tests - #13808
Conversation
Committed-By-Agent: codex Co-authored-by: codex <noreply@openai.com>
| val firstPane = onWebView().withElementByAnyTestId( | ||
| testIds = FINANCIAL_CONNECTIONS_LITE_INITIAL_PANE_TEST_IDS, | ||
| timeout = WEBVIEW_ELEMENT_TIMEOUT, | ||
| ) | ||
| firstPane.interaction.perform(webClick()) | ||
|
|
||
| onWebView() | ||
| .withElementByTestId("agree-button") | ||
| .withElementByTestId( | ||
| testId = FINANCIAL_CONNECTIONS_LITE_INITIAL_PANE_TEST_IDS.first { it != firstPane.testId }, | ||
| timeout = WEBVIEW_ELEMENT_TIMEOUT, | ||
| ) |
There was a problem hiding this comment.
From motivation:
The Financial Connections frontend flag bank_connections_enable_select_institution_first controls whether institution-default or agree-button is rendered first.
Instead of having special logic to handle either coming first, should the test itself specify which to wait for first based on that front end flag? Or maybe we should just make the tests deterministic rather than handling this flag being non-deterministic
There was a problem hiding this comment.
According to codex:
Server-owned bank_connections_enable_select_institution_first routing can place either pane first.
So I think this is right?
There was a problem hiding this comment.
Ah I see, I was misled by it being called a "frontend flag", I thought it was on the client. Seems ok then, though still unfortunate that we can't just have a deterministic environment for our tests to run in
amk-stripe
left a comment
There was a problem hiding this comment.
Approving because the webview timeouts seem good to have but I'm worried we're just obscuring a real issue in the tests rather than fixing it, regarding the non-determinism of which button appears first in the webview
Pull request was closed
Summary
Motivation
TestUSBankAccount#testUSBankAccountLiteSuccessintermittently failed after 30 seconds while waiting forinstitution-default.The Financial Connections frontend flag
bank_connections_enable_select_institution_firstcontrols whetherinstitution-defaultoragree-buttonis rendered first. The Android test driver hard-codedinstitution-defaultfollowed byagree-button, so runs assigned the opposite flow variant waited for an element that was not expected to exist on that pane.This fix works by polling both valid initial selectors during the same timeout. Once one appears, the driver records which selector matched, clicks it, and waits for the other pane. The cancellation flow similarly waits until either valid initial pane is ready before pressing Back. This follows the actual server-selected ordering instead of trying to hide the mismatch with a longer timeout.
Testing
Added tests
Modified tests
Manually verified
Temporarily wrapped
TestRuleswithShampooRule(20)and rantestUSBankAccountLiteSuccessonpixel2api33chrome: 20/20 iterations passed.Removed the temporary Shampoo rule and reran
testUSBankAccountLiteSuccessplustestUSBankAccountLiteCancelAllowsUserToContinue: 2/2 tests passed.git diff --checkScreenshots
Changelog
Not applicable; this only changes Android test infrastructure.