|
1 | 1 | # E2E Test Guidelines |
2 | 2 |
|
3 | | -Rules for writing reliable, non-flaky Playwright E2E tests in this monorepo. Derived from systematic fixes to 58+ spec files. |
| 3 | +Rules for writing reliable, non-flaky Playwright E2E tests in this monorepo. |
4 | 4 |
|
5 | 5 | ## Imports & Setup |
6 | 6 |
|
7 | 7 | Always use the custom fixtures, never raw Playwright: |
8 | 8 |
|
9 | 9 | ```javascript |
10 | 10 | import { test, expect } from "@mendix/run-e2e/fixtures"; |
11 | | -import { waitForMendixApp } from "@mendix/run-e2e/mendix-helpers"; |
12 | 11 | ``` |
13 | 12 |
|
14 | | -The custom fixture: |
| 13 | +Import helpers only when explicitly needed: |
15 | 14 |
|
16 | | -- Auto-wraps `page.goto()` to call `waitForMendixApp()` |
17 | | -- Manages worker-scoped Mendix sessions (1 per worker, auto-logout on teardown) |
18 | | -- No manual `afterEach` logout needed |
| 15 | +```javascript |
| 16 | +import { waitForDataReady } from "@mendix/run-e2e/mendix-helpers"; |
| 17 | +``` |
19 | 18 |
|
20 | | -## Waiting Strategies |
| 19 | +The custom fixture: |
21 | 20 |
|
22 | | -### Hierarchy (use the highest applicable level) |
| 21 | +- Auto-wraps `page.goto()` to call `waitForMendixApp()` — do NOT call it manually after `goto` |
| 22 | +- Worker-scoped sessions: 1 Mendix session per Playwright worker (4 in CI, 2 locally) |
| 23 | +- Auto-logout on teardown — no manual `afterEach` logout needed |
23 | 24 |
|
24 | | -1. `waitForMendixApp(page)` — session exists + no progress indicator + `.mx-page` rendered |
25 | | -2. `await expect(element).toBeVisible()` — specific element appeared |
26 | | -3. `await expect(rows).toHaveCount(N)` — data loaded with expected count |
27 | | -4. `waitForDataReady(page)` — opt-in ONLY when data sync timing genuinely matters |
| 25 | +## Waiting Strategies |
28 | 26 |
|
29 | | -### Banned Patterns |
| 27 | +Prefer web-first assertions over explicit waits — they auto-retry until timeout. |
30 | 28 |
|
31 | 29 | | Don't | Do Instead | Why | |
32 | 30 | | ------------------------------------------------ | ------------------------------------- | ---------------------------------------------------- | |
33 | 31 | | `page.waitForTimeout(N)` | Web-first assertion on expected state | Arbitrary delays: too short = flaky, too long = slow | |
34 | 32 | | `page.waitForLoadState("networkidle")` | `waitForMendixApp(page)` | Unrelated network traffic delays indefinitely | |
35 | 33 | | `page.waitForSelector(...)` then separate assert | `await expect(locator).toBeVisible()` | Combined wait+assert auto-retries | |
36 | 34 |
|
| 35 | +Use `waitForDataReady(page)` only when data sync timing genuinely matters. |
| 36 | + |
37 | 37 | ## Assertions |
38 | 38 |
|
39 | | -Always prefer Playwright web-first assertions — they auto-retry until timeout. |
| 39 | +Preferred: `toBeVisible`, `toHaveText`, `toHaveCount`, `toHaveCSS`, `toContainText`, `toHaveScreenshot`. |
40 | 40 |
|
41 | 41 | | Don't | Do Instead | Why | |
42 | 42 | | -------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------ | |
43 | 43 | | `const text = await el.allTextContents(); expect(text).toEqual(...)` | `await expect(locator).toContainText([...])` | Non-retrying snapshot vs auto-retrying | |
44 | 44 | | `await el.evaluate(el => el.getBoundingClientRect())` | `await expect(el).toHaveCSS("transform", "...")` | DOM inspection races vs CSS state assertion | |
45 | | -| `el.nth(1)` to disambiguate | More specific selector or wait first | nth() fragile to render order | |
46 | 45 | | `page.$$eval(...)` to extract data | `expect(locator).toContainText()` or `.toHaveText()` | evaluate snapshots DOM; locator assertions retry | |
47 | 46 |
|
48 | | -Preferred assertions: `toBeVisible`, `toHaveText`, `toHaveCount`, `toHaveCSS`, `toContainText`, `toHaveScreenshot`. |
49 | | - |
50 | 47 | ## Locator Patterns |
51 | 48 |
|
52 | | -| Don't | Do Instead | Why | |
53 | | -| --------------------------------------- | ------------------------------------------ | --------------------------------------- | |
54 | | -| `.nth(N)` on ambiguous selectors | `.mx-name-*` attribute selectors | nth fragile to DOM order | |
55 | | -| Complex CSS selectors | `page.locator(".mx-name-widgetName")` | mx-name attributes are stable, semantic | |
56 | | -| `page.click("text=...")` for navigation | `page.locator(".mx-name-navItem").click()` | Text fragile to i18n/copy changes | |
| 49 | +Prefer `.mx-name-*` attributes — set by Mendix Studio Pro from widget names, stable across DOM refactors and i18n changes. |
| 50 | + |
| 51 | +| Don't | Do Instead | Why | |
| 52 | +| ----------------------------------- | ----------------------------------------------- | -------------------------------------------- | |
| 53 | +| `.nth(N)` on ambiguous selectors | `.mx-name-*` attribute selectors | nth fragile to DOM order | |
| 54 | +| `page.click("text=...")` standalone | `.mx-name-*` or compose: CSS scope + role/label | Text alone = false positive, fragile to i18n | |
| 55 | +| Asserting text content in E2E | Unit/snapshot tests for text correctness | Text assertions belong in unit tests | |
| 56 | + |
| 57 | +When `.mx-name-*` is not available, compose locators — see [Playwright locator docs](https://playwright.dev/docs/locators): |
| 58 | + |
| 59 | +```javascript |
| 60 | +// mx-name — preferred |
| 61 | +page.locator(".mx-name-btnSubmit"); |
| 62 | + |
| 63 | +// composed: CSS scope + role |
| 64 | +page.locator(".mx-name-myForm").getByRole("button", { name: "Save" }); |
| 65 | + |
| 66 | +// composed: CSS scope + label |
| 67 | +page.locator(".mx-name-myWidget").getByLabel("Start date"); |
| 68 | +``` |
57 | 69 |
|
58 | 70 | ## Screenshot Testing |
59 | 71 |
|
60 | 72 | - No per-test `{ threshold: N }` or `{ maxDiffPixels: N }` overrides — use global config (`threshold: 0.1`) |
61 | 73 | - Always ensure element is visible before screenshot: `await expect(el).toBeVisible()` |
62 | 74 | - Animations disabled globally (`animations: "disabled"` + `reducedMotion: "reduce"`) |
63 | 75 |
|
64 | | -## Session Management |
65 | | - |
66 | | -- Worker-scoped sessions: 1 Mendix session per Playwright worker |
67 | | -- Workers: 4 in CI, 2 locally (stays under 5-session license limit) |
68 | | -- No manual `afterEach` logout — fixture handles cleanup |
69 | | -- No per-test browser context creation |
70 | | - |
71 | 76 | ## ESLint Enforcement |
72 | 77 |
|
73 | | -These rules are configured in `automation/run-e2e/eslint.config.mjs`: |
| 78 | +Configured in `automation/run-e2e/eslint.config.mjs`: |
74 | 79 |
|
75 | 80 | ``` |
76 | 81 | playwright/no-wait-for-timeout: error |
77 | 82 | playwright/no-networkidle: warn |
78 | 83 | playwright/prefer-web-first-assertions: warn |
79 | 84 | ``` |
80 | 85 |
|
81 | | -## Code Review Checklist |
82 | | - |
83 | | -- [ ] Uses `@mendix/run-e2e/fixtures` import (not `@playwright/test`) |
84 | | -- [ ] No `waitForTimeout` calls |
85 | | -- [ ] No `waitForLoadState("networkidle")` without explicit justification |
86 | | -- [ ] All assertions use web-first Playwright assertions |
87 | | -- [ ] No per-test screenshot threshold overrides |
88 | | -- [ ] No manual `afterEach` logout |
89 | | -- [ ] Locators use `.mx-name-*` attributes where possible |
90 | | -- [ ] Tests tagged `@smoke` if they cover critical paths |
91 | | - |
92 | 86 | ## Spec File Template |
93 | 87 |
|
94 | 88 | ```javascript |
95 | 89 | import { test, expect } from "@mendix/run-e2e/fixtures"; |
96 | | -import { waitForMendixApp } from "@mendix/run-e2e/mendix-helpers"; |
97 | 90 |
|
98 | 91 | test.describe("WidgetName", () => { |
99 | 92 | test.beforeEach(async ({ page }) => { |
100 | 93 | await page.goto("/"); |
101 | | - await waitForMendixApp(page); |
102 | 94 | }); |
103 | 95 |
|
104 | | - test("describes user-visible behavior", async ({ page }) => { |
| 96 | + test("describes user-visible behavior @smoke", async ({ page }) => { |
105 | 97 | // Arrange |
106 | 98 | await page.locator(".mx-name-navItem").click(); |
107 | | - await waitForMendixApp(page); |
108 | 99 |
|
109 | 100 | // Act |
110 | 101 | await page.locator(".mx-name-myWidget .some-input").fill("value"); |
|
0 commit comments