Skip to content

Commit 76bb745

Browse files
alex-strukclaude
andcommitted
fix(e2e): stop sending groupId on update; record the suite's rot (D-14)
Walking Part 4 led to running the workflow-builder e2e suite, which turned out to be the most valuable thing in this pass. first run: 14 failed, 51 passed after fix: 11 failed, 54 passed THREE OF THE FOURTEEN WERE MINE. f9049ab (G-063) gave UpdateWorkflowDto a whitelist so a save must state the version it was based on. The e2e helper had been updated to send expectedVersion but still sent groupId, which the whitelisted DTO now rejects with 400 "property groupId should not exist". A workflow's group is fixed at create time, so the helper was the wrong side. The other 11 are pre-existing and NOT triaged — 7 in tier2-control-flow (all the same selectNode timeout), 3 in tier2-port-wiring, 1 in tier2-validation. The product itself is fine: a genuine mouse click selects eachDoc/collect/ routeByType/pollOcr and opens each one's settings panel. This is test rot. Why it went unnoticed: .github/workflows/ has NO Playwright job. 76 specs exist and nothing runs them. That reframes the gap log — adding E1/E2 to a suite nobody runs buys very little. Wire the suite into CI first, then triage the 11, then add the new specs. My own regression sat there a day and was found only because I ran the suite by hand. Part 4 itself walks clean: all six control-flow types open their own settings panel, pollUntil carries the port rows G-016 gave it, and the condition editor renders a depth-3 AND/OR/NOT tree with a readable summary. Part 15 is not runnable here — no agent credentials in the environment (checked by presence only, never read). Its stubbed half passes via tier3-agent-stubbed. Ran with PLAYWRIGHT_SKIP_DB_RESET=1 throughout, so the dev DB was left alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent f0d4e60 commit 76bb745

4 files changed

Lines changed: 83 additions & 8 deletions

File tree

docs-md/workflows/MANUAL_TEST_PLAN.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -269,39 +269,39 @@ Each item is one control-flow node's settings form. **4.1–4.7** use the **firs
269269

270270
> **About the first demo workflow.** It is a **forms showcase, not a runnable pipeline** — one graph that packs all six control-flow node types so you can open each form in isolation. It maps over a `documents` array (**Run for each document**), and for each document a switch (**Branch by condition**) routes by type down one of three demo branches: *invoice* → a **Sub-workflow**, *receipt* → **Wait until condition** (poll) → **Extract OCR result**, and the *default* → **Wait for approval** (a human gate). A **join** (**Collect results**) would gather the per-document results into `results`, then **Store Results** persists them. **Two of the three branches (Sub-workflow, Wait for approval) deliberately dead-end** — they never reach the map body's exit node (**Extract OCR result**), so only the receipt branch completes a real chain. That is intentional here (the point is to inspect every form), but it would **not** be a valid production workflow — a real map body wants every branch to converge on the exit node. `documents`/`currentDoc` are **untyped trigger data** (no schema), so conditions on them (e.g. `currentDoc.type`) are typed by hand, not picked from a field list.
271271

272-
- [ ] **4.1 Switch — route each item down one edge.** *A switch tests its cases top-to-bottom; the first match sends the item down that case's **edge**, else the default edge. Key model: a case does **not** point at a target node directly — it points at a **conditional edge** that runs from this switch to a target. So a case is "condition → which outgoing edge to take."*
272+
- [x] **4.1 Switch — route each item down one edge.** *A switch tests its cases top-to-bottom; the first match sends the item down that case's **edge**, else the default edge. Key model: a case does **not** point at a target node directly — it points at a **conditional edge** that runs from this switch to a target. So a case is "condition → which outgoing edge to take."*
273273
1. Click **Branch by condition** (the yellow diamond) → its settings open on the right. It already has two cases plus a default, each bound to one conditional edge leaving the diamond (to **Sub-workflow**, to **Wait until condition**, and — as default — to **Wait for approval**).
274274
2. Click **Add Case** → a new empty case row appears. Click its **condition** → build a test (e.g. `currentDoc.type` **equals** `"invoice"`). *(`currentDoc` is an untyped loop item, so type the `.type` path by hand — see the untyped-data note above.)*
275275
3. Open the case's **Edge** dropdown. **What this dropdown is:** the outgoing edge this case fires. It lists **every conditional edge that starts at *this* switch** — normal edges, error edges, and edges leaving *other* switches are excluded, because a switch can only route down its own conditional branches. In this demo that's the three existing edges (Sub-workflow / Wait until condition / Wait for approval). **To route a new case somewhere *new*, first draw a new conditional edge** on the canvas from the switch's output handle to the target node; it then appears in this dropdown alongside the others. *(Note: the dropdown does not currently grey out edges already used by another case, so it's on you not to point two cases at the same edge.)*
276276
4. Optionally set a **Default edge** (taken when no case matches).
277277
**Pass:** the Edge dropdown lists this switch's conditional edges (and only those); **Save + reload** keeps every case and its edge.
278-
- [ ] **4.2 Map — run a body once per item.** *A map iterates a collection and runs its "body" sub-graph for each element; the body is delimited by an entry and exit node (the dashed green box on the canvas).*
278+
- [x] **4.2 Map — run a body once per item.** *A map iterates a collection and runs its "body" sub-graph for each element; the body is delimited by an entry and exit node (the dashed green box on the canvas).*
279279
1. Click **Run for each document** → its settings open.
280280
2. Set **collection ctx key** (`documents`), **item ctx key** (`currentDoc`), optional **index ctx key**, and **max concurrency** (≥ 1).
281281
3. Set **body entry node** and **body exit node** via their pickers (each excludes the map node itself).
282282
**Pass:** all fields are editable; the green **body box** wraps the nodes between the entry and exit you chose.
283-
- [ ] **4.3 Join — collect a map's results.** *A join gathers the per-item outputs a map produced back into one array.*
283+
- [x] **4.3 Join — collect a map's results.** *A join gathers the per-item outputs a map produced back into one array.*
284284
1. Click **Collect results** → its settings open.
285285
2. Open the **Source Map node** picker → it lists **only map nodes** (not activities/switches).
286286
3. Set the **Results ctx key** (where the collected array is written).
287287
**Pass:** only map nodes are selectable as the source. *(Join strategy is fixed to `all` and isn't shown in the form.)*
288-
- [ ] **4.4 Sub-workflow (childWorkflow) — call another graph.** *Runs a whole other workflow — either a saved Library workflow or an inline graph shipped with this node.*
288+
- [x] **4.4 Sub-workflow (childWorkflow) — call another graph.** *Runs a whole other workflow — either a saved Library workflow or an inline graph shipped with this node.*
289289
1. Click **Sub-workflow (inline OCR)** → its settings open.
290290
2. Toggle **Library / Inline**. In **Inline** mode an **editable** JSON textarea of the embedded child graph shows, with a live problems list beneath it (see 5.7).
291291
3. Edit the **input** and **output mapping** lists (which parent ctx keys feed the child, and where its outputs land).
292292
**Pass:** the Library/Inline toggle switches modes; inline shows the JSON editor and, under it, either *"No problems in the inline graph."* or a red/amber panel listing them. *(Library-picker modal is a manual spot-check; inline round-trip is `tier2-control-flow` e2e.)*
293-
- [ ] **4.5 Wait until condition (pollUntil) — poll until true.** *Re-runs an activity on an interval until a condition holds (or it times out).*
293+
- [x] **4.5 Wait until condition (pollUntil) — poll until true.** *Re-runs an activity on an interval until a condition holds (or it times out).*
294294
1. Click **Wait until condition** → its settings open.
295295
2. Pick an **activity type** → its own parameters sub-form renders below.
296296
3. Set the **condition**, an **interval** (e.g. `30s`), **max attempts**, and optional **initial delay** / **timeout**.
297297
4. Type an invalid duration (e.g. `30` with no unit) in a duration field.
298298
**Pass:** the invalid duration shows an inline error; the picked activity's nested params render. *(Interval / invalid-duration / maxAttempts round-trips are `tier2-control-flow` e2e; the activity sub-form is a manual spot-check.)*
299-
- [ ] **4.6 Wait for approval (humanGate) — pause for a human.** *Suspends the run until a named signal arrives, with a timeout fallback.*
299+
- [x] **4.6 Wait for approval (humanGate) — pause for a human.** *Suspends the run until a named signal arrives, with a timeout fallback.*
300300
1. Click **Wait for approval** → its settings open.
301301
2. Note the **signal name**, the read-only **payload schema**, and the required **timeout**.
302302
3. Set **On timeout** to `fail`, then `continue`, then `fallback`.
303303
**Pass:** the **fallback edge** picker appears **only** when On timeout = `fallback` (gone for `fail`/`continue`).
304-
- [ ] **4.7 Condition editor — nested expressions.** *Builds boolean trees (AND/OR/NOT + comparisons) to any depth; each value is a **Ref** (a variable) or a **Literal**.*
304+
- [x] **4.7 Condition editor — nested expressions.** *Builds boolean trees (AND/OR/NOT + comparisons) to any depth; each value is a **Ref** (a variable) or a **Literal**.*
305305
1. On **Branch by condition**, open the **second case**'s condition — this demo ships a 3-level tree `AND( OR(EQ, GTE), NOT(IS-NULL) )`.
306306
2. Confirm the nesting renders indented per level.
307307
3. Pick a leaf value and toggle it **Ref****Literal** (Ref = variable autocomplete; Literal = plain input).

feature-docs/20260724-workflow-builder-spec-completion/COVERAGE_GAP_LOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,17 @@ cannot* see the failure.
5050
| **E1** | `tier1-dynamic-node`**system-admin identity** variant | the `/dynamic-nodes` breakage (`dd6cdafb`) | The one that actually bit us. Unit tests mock `fetch` and never assert what reaches the wire; the `x-api-key` path resolves to exactly one group, so **no** existing layer reaches the admin branch. Needs the request-level admin simulation recorded in the `app-browser-auth` skill | S |
5151
| **E2** | `tier1-dynamic-node` — publish from the palette modal drops the node | 14.8 auto-drop (`64d86d73`) | The decisive argument: my unit test passes, but the live bug had a **second** cause (React render timing) the unit test did not model. I only found it in a browser | S |
5252

53+
### ⚠️ G2 is now second priority — see D-14
54+
55+
`.github/workflows/` contains **no Playwright job**. 76 e2e specs exist and
56+
nothing runs them, so **11 specs are failing today** and one of the failures was
57+
a regression I introduced a day earlier (`f9049ab3` whitelisted
58+
`UpdateWorkflowDto`; the e2e helper still sent `groupId`). It was caught only
59+
because I ran the suite by hand during this walk.
60+
61+
Adding E1/E2 to a suite nobody runs buys very little. **Wire the existing suite
62+
into CI first**, then triage the 11 pre-existing failures, then add E1/E2.
63+
5364
**Explicitly not adding e2e for:** undo-during-replay (`fd3194bb`) and the 9.9c
5465
replay chip. Both are fully pinned by falsifiable unit tests; an e2e would be
5566
duplicate cost.

feature-docs/20260724-workflow-builder-spec-completion/WALKTHROUGH_PARTS_2_14.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,3 +250,65 @@ Running tally of first-pass "failures" that were my technique, not the product:
250250
D-11 timing, `force: true` on a disabled item, Compare-to-head on the head row,
251251
and this. **Any first-pass failure is now unverified until re-probed a second
252252
way.**
253+
254+
---
255+
256+
# Part 4, the e2e suite, and Part 15
257+
258+
## Part 4 — control-flow settings forms
259+
260+
Walked manually on the Part-4 demo (which carries all six control-flow node
261+
types plus a map body).
262+
263+
| # | Verdict | Evidence |
264+
|---|---|---|
265+
| 4.1–4.6 forms | ✅ PASS | every type opens its own panel on a genuine click: `switch-node-settings`, `map-node-settings`, `join-node-settings` (`source-map-node-id`, `results-ctx-key`), `child-workflow-node-settings` (`ref-type`, `inline-editor`, `inline-problems-none`, `input-row-0`), `poll-until-node-settings` (+ `poll-until-wrapped-pollOcr` and full port rows — G-016's "wrappers inherit affordances"), `human-gate-node-settings` (`signal-name`, `payload-schema-editor`, `timeout`) |
266+
| 4.7 Nested conditions | ✅ PASS | depth-3 tree — `case-1-condition-operand-0-editor-operand-0-editor-left-ref-input` — with the readable summary *"ALL OF THESE MUST BE TRUE — (ctx.currentDoc.type is "receipt" or ctx.currentDoc.confidence ≥ 0.8) and not (…)"* |
267+
| 4.8–4.15 | ◑ e2e-backstopped | `tier2-condition-step-ref` covers the step-picker/persistence half; 4.12 and 4.15 need a run |
268+
269+
## D-14 — the e2e suite has rotted, and nothing runs it (open)
270+
271+
Running the workflow-builder suite (`PLAYWRIGHT_SKIP_DB_RESET=1`, so Alex's dev
272+
DB was left alone):
273+
274+
```
275+
first run: 14 failed, 51 passed
276+
after fix: 11 failed, 54 passed
277+
```
278+
279+
**Three of the fourteen were mine, and I fixed them.** `f9049ab3` (G-063) gave
280+
`UpdateWorkflowDto` a whitelist so a save must state the version it was based
281+
on. The e2e helper had been updated to send `expectedVersion` but still sent
282+
`groupId`, which the whitelisted DTO now rejects:
283+
284+
```
285+
update workflow failed: 400 {"message":["property groupId should not exist"]}
286+
```
287+
288+
A workflow's group is fixed at create time, so the helper was the wrong side —
289+
`groupId` removed from the PUT payload.
290+
291+
**The other 11 are pre-existing and un-triaged**: 7 in `tier2-control-flow`
292+
(all the same `selectNode` timeout), 3 in `tier2-port-wiring`, 1 in
293+
`tier2-validation`. I did **not** establish their cause. What I did establish is
294+
that **the product works**: a genuine `page.mouse.click` selects `eachDoc`,
295+
`collect`, `routeByType` and `pollOcr` and opens `map-node-settings`,
296+
`join-node-settings`, `switch-node-settings` and `poll-until-node-settings`
297+
respectively. So this is test rot, not broken selection.
298+
299+
**The reason it went unnoticed: `.github/workflows/` has no Playwright job at
300+
all.** 76 e2e specs exist and nothing runs them. That reframes gap **G2** — E1
301+
and E2 are worth much less than wiring the existing suite into CI, because a
302+
spec nobody runs is a spec that silently rots. My own regression sat there for a
303+
day and was caught only because I happened to run the suite by hand.
304+
305+
Recommended order: **wire e2e into CI first**, then triage the 11, then add
306+
E1/E2.
307+
308+
## Part 15 — AI agent ☁️🔑
309+
310+
**NOT RUNNABLE HERE.** No agent credentials in the environment
311+
(`ANTHROPIC_API_KEY`, `AZURE_OPENAI_*` all unset — checked by presence only,
312+
never read). The stubbed half is covered by `tier3-agent-stubbed`, which passes
313+
in the suite run above. The live half (15.x against a real model) is manual-only
314+
and needs Alex.

tests/e2e/workflow-builder/helpers/workflow-api.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,10 +133,12 @@ export async function updateWorkflow(
133133
opts.expectedVersion ?? (await getWorkflow(request, id)).version;
134134
const res = await request.put(`${BACKEND_URL}/api/workflows/${id}`, {
135135
headers,
136+
// No `groupId`: a workflow's group is fixed at create time, and
137+
// `UpdateWorkflowDto` is whitelisted, so sending it is a 400
138+
// ("property groupId should not exist") rather than a no-op.
136139
data: {
137140
name: opts.name,
138141
config: opts.config,
139-
groupId: SEED_GROUP_ID,
140142
expectedVersion,
141143
},
142144
});

0 commit comments

Comments
 (0)