[draft] ui testing by scottgerring · Pull Request #154 · DataDog/stickerlandia

scottgerring · 2026-01-19T15:31:13Z

Adds a basic playwright test suite, hooks it up to our compose integration test in github, as well as running after each service completes deploying to AWS on main.

The suite can be run both for compose (mise compose:ui-test) and AWS environments (mise aws:ui-test).

I have also made a small change to user-management to improve its rate limiting.

Test results are annotated onto the actions. Tests that fail and then succeed are annotated as flaky, but the build continues - see e.g. this one here. Playwright tests add about 30s on top of that build; not much when so much time is sunk into the container builds themselves beforehand.

scottgerring · 2026-01-20T10:20:20Z

@@ -2,6 +2,7 @@ name: E2E Tests



This runs the suite after every service completes its AWS deploy. I think this is a bit OTT, but I can't see an easy way of saying "when all the deploys are done", without having one big mega-job that everything is tied into.

Open to suggestions here.

I think this works, how long do the tests take to run? I assume not very long.

scottgerring · 2026-01-20T10:20:39Z

@@ -0,0 +1,158 @@
+import { test as base, expect, type Page } from '@playwright/test';


A fixture to get us into a logged in state

scottgerring · 2026-01-20T10:20:45Z

@@ -0,0 +1,209 @@
+import { Page, expect, Response } from '@playwright/test';


Test we can login

This is very cool that you can test the whole OAUth login flow :) Nice work.

scottgerring · 2026-01-20T10:22:45Z

@@ -0,0 +1,32 @@
+import { test, expect } from '@playwright/test';


Basic API endpoint tests. \

scottgerring · 2026-01-20T10:23:00Z

@@ -0,0 +1,123 @@
+import { test, expect } from '@playwright/test';


The actual "can i login" test

scottgerring · 2026-01-20T10:23:17Z

@@ -0,0 +1,40 @@
+import { test, expect } from '@playwright/test';


A stubbed out logout test until #148

scottgerring · 2026-01-20T10:25:04Z

@@ -0,0 +1,136 @@
+import { test, expect } from '@playwright/test';


This needs #157 so it can do cleanup afterwards. as long as we are running against dev only, I don't really care (and it produces interesting event traffic!)

…ent IP

scottgerring · 2026-01-20T10:28:38Z


 builder.Services.AddRateLimiter(options =>
 {
+    // Log when requests are rejected due to rate limiting


This isn't related, except that I was getting 503s from throttling a bunch with the test suite.

Now we are:

Rate limiting less aggressively, and per-client

Returning a 429 and logging to say what's happened

Trying to track client honouring forward-for headers

scottgerring · 2026-01-20T10:30:08Z

+
+        context.HttpContext.Response.Headers.RetryAfter = retryAfter.ToString();
+
+        // Get client IP from X-Forwarded-For (behind LB) or RemoteIpAddress


We go to lengths to try use forwarded headers for identifying actual client (and not LB)

jeastham1993 · 2026-01-20T10:46:40Z

@@ -2,6 +2,7 @@ name: E2E Tests



I think this works, how long do the tests take to run? I assume not very long.

jeastham1993 · 2026-01-20T10:47:22Z

@@ -0,0 +1,209 @@
+import { Page, expect, Response } from '@playwright/test';


This is very cool that you can test the whole OAUth login flow :) Nice work.

jeastham1993 · 2026-01-20T10:48:07Z

+  readonly backButton: Locator;
+  readonly shareButton: Locator;
+
+  constructor(page: Page) {


@scottgerring this looks like the kind of thing that would be really really flaky. Relying on data and h1/h2's. Any thoughts on how we prevent that?

so for real flaky, they retry if they flake and flag it on the github UI that it was flaky, but still succeed. they only fail build if the retries fail too.

if you change the UI and break the tests then your build will break like anything else in github (and you can easily check locally - it takes about 8s to run the suite now via mise compose:ui-test).

matching on elements - i think thats just the cost of doing business for this stuff!

scottgerring force-pushed the ui-testing branch 5 times, most recently from 473be10 to 18d5318 Compare January 20, 2026 10:16

scottgerring commented Jan 20, 2026

View reviewed changes

Comment thread .github/workflows/docker-compose-integration.yml

scottgerring commented Jan 20, 2026

View reviewed changes

scottgerring added 2 commits January 20, 2026 11:26

feat(e2e): start implementing E2E tests

3cbf8ef

chore(user-management): relax rate limiting, log it, and do it by cli…

9f0dd6d

…ent IP

scottgerring force-pushed the ui-testing branch from 18d5318 to 36bdf98 Compare January 20, 2026 10:26

scottgerring commented Jan 20, 2026

View reviewed changes

scottgerring marked this pull request as ready for review January 20, 2026 10:34

scottgerring added the ui-test label Jan 20, 2026

scottgerring requested a review from jeastham1993 January 20, 2026 10:35

jeastham1993 approved these changes Jan 20, 2026

View reviewed changes

scottgerring force-pushed the ui-testing branch from 36bdf98 to c4c225d Compare January 20, 2026 10:54

chore(ui-testing): Trigger in workflow

d50e132

scottgerring force-pushed the ui-testing branch from c4c225d to d50e132 Compare January 20, 2026 11:54

scottgerring merged commit fdd9b5d into main Jan 20, 2026
24 checks passed

scottgerring deleted the ui-testing branch January 20, 2026 12:04

scottgerring mentioned this pull request Jan 21, 2026

feat(ui-test): Create a basic UI testing framework to increase stability confidence #145

Closed

scottgerring added the stabilisation label Jan 22, 2026

		@@ -0,0 +1,158 @@
		import { test as base, expect, type Page } from '@playwright/test';

		@@ -0,0 +1,209 @@
		import { Page, expect, Response } from '@playwright/test';

		@@ -0,0 +1,32 @@
		import { test, expect } from '@playwright/test';

		@@ -0,0 +1,123 @@
		import { test, expect } from '@playwright/test';

		@@ -0,0 +1,40 @@
		import { test, expect } from '@playwright/test';

		@@ -0,0 +1,136 @@
		import { test, expect } from '@playwright/test';


		context.HttpContext.Response.Headers.RetryAfter = retryAfter.ToString();

		// Get client IP from X-Forwarded-For (behind LB) or RemoteIpAddress

Uh oh!

Conversation

scottgerring commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

scottgerring commented Jan 19, 2026 •

edited

Loading