|
| 1 | +# Writing UX Tests (Cypress) |
| 2 | + |
| 3 | +Cypress specs in `web/cypress/e2e/` verify the React web UI end-to-end |
| 4 | +against a running Sympozium cluster. They cover wizard flows, list/detail |
| 5 | +pages, run dispatching, deletion, persona-pack lifecycle, and regression |
| 6 | +guards for UX-visible bugs. |
| 7 | + |
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +- Kind (or other) cluster with Sympozium installed (`make install`). |
| 11 | +- `kubectl` pointing at that cluster. |
| 12 | +- Node dependencies installed: `make web-install` (or `cd web && npm install`). |
| 13 | +- Optional but recommended for live LLM scenarios: **LM Studio running at |
| 14 | + `host.docker.internal:1234`** with at least one model loaded (the |
| 15 | + default specs target `qwen/qwen3.5-9b`). |
| 16 | + |
| 17 | +## Running the Tests |
| 18 | + |
| 19 | +There are two supported flows — pick whichever matches the server you |
| 20 | +have running: |
| 21 | + |
| 22 | +### A) Vite dev server flow |
| 23 | + |
| 24 | +Best for active frontend development with hot reload. |
| 25 | + |
| 26 | +```bash |
| 27 | +make web-dev-serve # terminal 1: vite on :5173 + port-forward apiserver |
| 28 | +make ux-tests # terminal 2: headless Cypress run |
| 29 | +make ux-tests-open # …or launch the interactive Cypress runner |
| 30 | +``` |
| 31 | + |
| 32 | +### B) `sympozium serve` flow |
| 33 | + |
| 34 | +Best if you already have `sympozium serve` running against an installed |
| 35 | +cluster (serves the embedded UI from the in-cluster apiserver). |
| 36 | + |
| 37 | +```bash |
| 38 | +sympozium serve # terminal 1: port-forwards apiserver → :9090 |
| 39 | +make ux-tests-serve # terminal 2: headless, against :9090 |
| 40 | +make ux-tests-serve-open # …or interactive |
| 41 | +``` |
| 42 | + |
| 43 | +If you run `sympozium serve --port 8090`, pass the matching port to make: |
| 44 | + |
| 45 | +```bash |
| 46 | +make ux-tests-serve SERVE_PORT=8090 |
| 47 | +``` |
| 48 | + |
| 49 | +Both targets auto-retrieve the API token from the `sympozium-ui-token` |
| 50 | +secret in the `sympozium-system` namespace and wire it into Cypress via |
| 51 | +`CYPRESS_API_TOKEN`. |
| 52 | + |
| 53 | +## What Gets Tested |
| 54 | + |
| 55 | +| Area | Spec | |
| 56 | +|---|---| |
| 57 | +| Instance wizard (adhoc + LM Studio) | `instance-create-adhoc.cy.ts`, `instance-create-lmstudio.cy.ts`, `instance-multi-run-lmstudio.cy.ts` | |
| 58 | +| PersonaPack activation + lifecycle | `personapack-enable.cy.ts`, `personapack-full-lifecycle.cy.ts`, `personapack-channel-bind.cy.ts` | |
| 59 | +| Run detail regression guards | `run-detail-response-visible.cy.ts`, `run-thinking-indicator.cy.ts` | |
| 60 | +| Deletion flows | `instance-delete-with-running-runs.cy.ts`, `run-delete-and-disappear.cy.ts`, `schedule-delete.cy.ts` | |
| 61 | +| Runs list | `runs-filter-and-sort.cy.ts` | |
| 62 | +| Schedules | `schedule-create-via-ui.cy.ts`, `schedule-pause-resume.cy.ts` | |
| 63 | +| Auxiliary pages | `skills-catalog.cy.ts`, `policies-view.cy.ts`, `mcp-server-add.cy.ts`, `login-flow.cy.ts` | |
| 64 | + |
| 65 | +## Writing a New Spec |
| 66 | + |
| 67 | +Specs live in `web/cypress/e2e/<name>.cy.ts`. The support file at |
| 68 | +`web/cypress/support/e2e.ts` provides shared helpers that remove a lot of |
| 69 | +boilerplate: |
| 70 | + |
| 71 | +| Helper | Purpose | |
| 72 | +|---|---| |
| 73 | +| `cy.createLMStudioInstance(name)` | POST to `/api/v1/instances` with an LM Studio + qwen3.5-9b config | |
| 74 | +| `cy.dispatchRun(instanceRef, task)` | POST to `/api/v1/runs`; resolves with the created AgentRun name | |
| 75 | +| `cy.waitForRunTerminal(runName)` | Polls `/api/v1/runs/:name` until `status.phase` is `Succeeded` or `Failed` | |
| 76 | +| `cy.waitForDeleted(path)` | Polls until a GET returns 404 (handles finalizer delays) | |
| 77 | +| `cy.deleteInstance(name)` / `cy.deleteRun(name)` / `cy.deleteSchedule(name)` / `cy.deletePersonaPack(name)` | API-level cleanup helpers | |
| 78 | +| `cy.wizardNext()` / `cy.wizardBack()` | Click Next/Back buttons in onboarding wizards | |
| 79 | + |
| 80 | +Minimal template for a live-cluster spec: |
| 81 | + |
| 82 | +```ts |
| 83 | +const INSTANCE = `cy-myspec-${Date.now()}`; |
| 84 | +let RUN = ""; |
| 85 | + |
| 86 | +describe("My feature", () => { |
| 87 | + before(() => { |
| 88 | + cy.createLMStudioInstance(INSTANCE); |
| 89 | + }); |
| 90 | + |
| 91 | + after(() => { |
| 92 | + if (RUN) cy.deleteRun(RUN); |
| 93 | + cy.deleteInstance(INSTANCE); |
| 94 | + }); |
| 95 | + |
| 96 | + it("does the thing", () => { |
| 97 | + cy.dispatchRun(INSTANCE, "reply: HELLO").then((name) => { |
| 98 | + RUN = name; |
| 99 | + }); |
| 100 | + cy.then(() => cy.waitForRunTerminal(RUN)); |
| 101 | + cy.visit(`/runs/${RUN}`); |
| 102 | + cy.contains("HELLO", { timeout: 20000 }).should("be.visible"); |
| 103 | + }); |
| 104 | +}); |
| 105 | + |
| 106 | +export {}; |
| 107 | +``` |
| 108 | + |
| 109 | +### Conventions |
| 110 | + |
| 111 | +- **End every spec with `export {};`** so each file is a TS module and |
| 112 | + its top-level `const`s don't collide with sibling specs under `tsc`. |
| 113 | +- **Use lowercase resource names** — Kubernetes rejects uppercase in |
| 114 | + object names (RFC 1123 subdomain rules). |
| 115 | +- **Prefer `cy.waitForDeleted()` over direct 404 assertions** — finalizers |
| 116 | + can delay GC and a naïve `expect(200).to.eq(404)` is racey. |
| 117 | +- **For operations without an apiserver endpoint** (e.g. schedule |
| 118 | + suspend, PersonaPack creation), fall back to `cy.exec("kubectl ...")` |
| 119 | + via manifests written with `cy.writeFile("cypress/tmp/…yaml", …)`. |
| 120 | +- **Don't block on the "thinking" indicator inside tight loops** — short |
| 121 | + tasks may finish before Cypress can observe the transient phase. |
| 122 | + |
| 123 | +### Token Injection |
| 124 | + |
| 125 | +`web/cypress/support/e2e.ts` overrides `cy.visit` to set |
| 126 | +`localStorage.sympozium_token` from `CYPRESS_API_TOKEN` before the app |
| 127 | +boots, so your specs can assume the user is authenticated. If you need |
| 128 | +to test the unauthenticated path (see `login-flow.cy.ts`), pass an |
| 129 | +`onBeforeLoad` hook that calls `win.localStorage.removeItem( |
| 130 | +"sympozium_token")` — it runs after the token-injecting override, so |
| 131 | +your removal wins. |
| 132 | + |
| 133 | +## Troubleshooting |
| 134 | + |
| 135 | +### `Error: Cannot find module 'cypress'` |
| 136 | + |
| 137 | +Cypress is in `package.json` but `node_modules/` is stale. Fix: |
| 138 | + |
| 139 | +```bash |
| 140 | +make web-install |
| 141 | +# or: cd web && npm install |
| 142 | +``` |
| 143 | + |
| 144 | +### `nothing is listening on localhost:<port>` |
| 145 | + |
| 146 | +The preflight check (`hack/check-ux-backend.sh`) couldn't reach |
| 147 | +`/api/v1/namespaces` at the expected port. Either no dev server is up, |
| 148 | +or a previous port-forward died and its local listener is still held |
| 149 | +by a zombie process: |
| 150 | + |
| 151 | +```bash |
| 152 | +# inspect who is on the port |
| 153 | +lsof -i :5173 -P -n # vite |
| 154 | +lsof -i :9090 -P -n # sympozium serve |
| 155 | +# kill if stale |
| 156 | +kill $(lsof -t -i :5173) 2>/dev/null || true |
| 157 | +``` |
| 158 | + |
| 159 | +### LM Studio-dependent specs hang |
| 160 | + |
| 161 | +Some specs dispatch real AgentRuns (`run-detail-*`, `instance-delete-*`, |
| 162 | +etc.). They need LM Studio reachable from inside Kind as |
| 163 | +`host.docker.internal:1234` and a `NetworkPolicy` that allows egress on |
| 164 | +port 1234. If your agent pods are failing to reach LM Studio, the |
| 165 | +integration test `test/integration/test-lmstudio-response-regression.sh` |
| 166 | +has the exact NetworkPolicy patch you need. |
| 167 | + |
| 168 | +### "name, provider, and model are required" |
| 169 | + |
| 170 | +Your helper is sending the wrong JSON shape. The apiserver's |
| 171 | +`POST /api/v1/instances` expects flat top-level fields: |
| 172 | + |
| 173 | +```json |
| 174 | +{ "name": "…", "provider": "lm-studio", "model": "…", "baseURL": "…" } |
| 175 | +``` |
| 176 | + |
| 177 | +Use `cy.createLMStudioInstance(name)` which handles this correctly. |
| 178 | + |
| 179 | +## Adding a New Helper |
| 180 | + |
| 181 | +Extend `web/cypress/support/e2e.ts`: |
| 182 | + |
| 183 | +```ts |
| 184 | +declare global { |
| 185 | + namespace Cypress { |
| 186 | + interface Chainable { |
| 187 | + myHelper(arg: string): Chainable<void>; |
| 188 | + } |
| 189 | + } |
| 190 | +} |
| 191 | + |
| 192 | +Cypress.Commands.add("myHelper", (arg: string) => { |
| 193 | + // … |
| 194 | +}); |
| 195 | +``` |
| 196 | + |
| 197 | +Run `cd web/cypress && npx tsc --noEmit` to type-check. |
0 commit comments