E2E enhancements

[jbordonado]

Problem

Flaky E2E tests are slowing down dev teams and interrupt dev process in case of false alerts.

Objective

Introduce shortcuts and mocks to improve velocity and stability of E2E tests.

Tasks

Preview Give feedback

1. [E2E tests] Add wrappers to simulate UI flows programmatically #8032
   Enhancement category: e2e focus: devops +3
2. [E2E tests] Introduce a set of smoke tests for the plugin #8031
   category: e2e focus: devops type: enhancement +3
3. https://github.com/Automattic/woocommerce-payments-meta/issues/83
4. https://github.com/Automattic/woocommerce-payments-meta/issues/82
5. Enhance the local E2E testing setup #7495
   category: e2e focus: devops type: enhancement +3
   
6. Timeout errors on E2E Tests on Atomic - All #7286
   category: e2e focus: devops type: developer experience type: enhancement +4

[vbelolapotkov]

Added two more tasks to this one based on discussion here: paJDYF-cc7-p2#comment-23044

[haszari]

Might need a coordinated program for improving our e2e setup, not clear if this project is aligned with or competing with moving to Playwright (https://github.com/Automattic/woocommerce-payments-meta/issues/330)

[bborman22]

Similar to @haszari's question, I put a thumbs down on this one only because I feel like we would get more benefit from the conversion to Playwright. Though I don't think it has to be one or the other, I do think the Playwright conversion should be first priority and some of these additional enhancements might become less urgent.

[vbelolapotkov]

Good comments @haszari @bborman22 .

not clear if this project is aligned with or competing with moving to Playwright

No, I don't think it's competing with it, and I think they are aligned. Brian made a good point though, it shouldn't be either or, but the scope and focus of each might change depending on which one goes first. And based on the voting results, Playwright migration resonates better with the group.

I feel like we would get more benefit from the conversion to Playwright.

That's a fair feeling, and thanks for taking it into account while voting.