Commit 20c6c0a
authored
ci(e2e): shard the full UI functional pool into a merge-ready matrix (#5883)
* ci: benchmark deduplicated UI merge sentinel
* ci: benchmark restoration-hosted sentinel
* ci: benchmark distributed P0 sentinel
* ci: benchmark split P0 sentinel
* ci: serialize split sentinel benchmark
* ci: finalize deduplicated UI merge topology
* ci: prototype all-shades UI domain topology
* ci: bound full UI runtime by shade
* test(e2e): stabilize full UI suite
* fix(web): unclip DS picker popover and propagate patched project metadata
Two regressions surfaced by re-enabling the full UI pool:
- The New Project design-system popover lost its portal and was clipped by
short viewports and the modal body. Render it through createPortal with a
viewport-aware anchor that flips upward when space below runs out.
- After a working-dir pick, onProjectMetadataChange forwarded only the
metadata onto a stale project copy, so ProjectView.currentProject's
recency comparison let an older useProjectDetail snapshot shadow the
accepted PATCH and the picker label never updated. The callback now
carries the daemon's authoritative post-patch project (fresh updatedAt
included), matching the design-system switch path, and ProjectView hands
it straight to onProjectChange.
* test(e2e): repair remaining stale full-suite UI contracts
Second repair batch for the failures surfaced by full diagnostic run
29480362163, plus one CI-invisible case that was serially skipped behind
the BYOK failure in the same file:
- Point the working-dir metadata spec at a directory that exists on the
runner instead of a hardcoded absolute path, and assert the label from
the picked path (this spec is red without the ProjectView patched-project
fix).
- Navigate back to All project files before asserting the plugin-folder
cards, since a finished run auto-opens the produced file tab.
- Resolve the create-run response race in real-daemon-run's sendPrompt via
a single waitForResponse alongside the click.
- Refresh selectors and copy for entry topbar, home hero rail, chat todo
autoscroll, project management, workspace keyboard, design files, manual
edit, restoration, and DS picker clipping flows to match the current UI
contracts while keeping each spec's oracles.
* ci: distribute the full UI suite over a generic 8-shard matrix
Replace the diagnostic four-domain P0/P1/P2 serial topology with a single
all-shades Playwright invocation per shard (--shard=N/8, two fully
parallel workers, 20-minute cap) and drop the scheduled trigger; full runs
stay manual until the merge-queue wiring lands. Update the workflow
topology guard accordingly.
* ci: serialize full UI shards and widen the matrix to twelve
The eight-shard two-worker experiment left six light shards healthy
(260-477s test steps) but both shards holding runtime-heavy files
degraded: repeated two-minute timeouts with retry recovery in one, and a
consistent create-run response starvation failure in the other. This
matches the earlier finding that runtime-heavy scenarios cannot share a
runner under two fully parallel workers. Run each shard with a single
worker and rebalance to twelve shards; fully-parallel stays on only for
test-granularity sharding.
* fix(web): preserve prompt-extracted plugin inputs across apply reconcile
The active-plugin apply reconcile rebuilt the inputs from the bind-time
optimistic snapshot, so edits the user typed during the apply roundtrip
(already extracted into active.inputs by handlePromptChange) were silently
reverted to defaults — the submitted pluginInputs then disagreed with the
visible prompt. Merge the apply result's defaults into the current inputs
inside the functional updater instead, mirroring the guard the prompt-text
reconcile already had.
* test(e2e): stabilize home gallery flows for all-shades execution
Generic all-shades sharding executes tests in orders the shade-filtered
domain topology never produced, exposing a home-surface remount window:
the async projects fetch can mount or remount HomeTemplatesReveal after a
test starts interacting, collapsing the gallery (inert) under the test's
cursor. Stabilize the shared helpers instead of individual specs:
- gotoEntryHome arms a projects-list response waiter before navigating and
holds until it settles.
- revealHomeTemplates requires the revealed state (or a true pass-through)
to survive a settle window, retrying when the wrapper mounts late; a
card's isVisible() is true inside the collapsed container, so
openHomePluginDetails now always routes through the reveal path.
- The browse-registry spec reveals before clicking.
- The DS-picker options moved out of the new-project-modal subtree with the
portal fix; query the page-level popover per the clipping spec's
convention.
- real-daemon-run's sendPrompt reports on timeout whether POST /api/runs
was ever issued, so the CI-only BYOK failure can be split into
swallowed-click vs unanswered-request on the next reproduction.
* test(e2e): make BYOK availability hermetic and reveal remaining gallery clicks
The BYOK missing-provider spec could only pass on machines with a real
`opencode` binary: daemon detection maps byok-opencode's availability
through the `opencode` agent's configured cli env, and the web composer
refuses to POST /api/runs for an unavailable agent (the sendPrompt
diagnostic confirmed "was never issued" on CI). Point the helper's
agentCliEnv at the fake opencode runtime — the run still fails pre-spawn
on the missing provider config, so the fake binary never executes.
Also route the two remaining collapsed-container gallery clicks (the
browse-registry starter journey and the detail-use force-click) through
revealHomeTemplates; an inert container swallows force-clicks silently.
* ci: probe a 24-shard full UI matrix against the four-minute wall target
The first green 12-shard run put eleven shards at 141-354s test steps but
concentrated the contiguous runtime-heavy chain in one 894s shard, setting
a ~16-minute wall tail. With ~3568 total test-seconds and ~90s per-job
overhead, 24 shards should land every job near four minutes.
* ci: settle the full UI matrix on twelve shards
The 24-shard probe stayed green but proved the runtime-heavy chain is a
serial group (test.describe.configure serial in real-daemon-run), which
Playwright keeps atomic within one shard: the tail stayed ~16 minutes while
the extra twelve cold starts raised aggregate runner time from ~76 to ~102
minutes. Twelve shards dominate; lowering the wall below the serial-chain
floor is a de-serialization follow-up, not a shard-count knob.
* test(e2e): drop serial mode from shard-splittable files
Playwright keeps a serial group atomic within one shard, so
real-daemon-run's file-level serial made its ~16-minute chain the floor of
the full UI matrix wall regardless of shard count. Every test in the three
serial files already performs its full setup per test (config reset,
route stubs, own project), so serial only added skip-after-failure and
shard atomicity. Verified order independence by running each file's two
standalone shard halves locally (fully parallel sharding, one worker):
21/21, 57/57, and 12/12 across the halves.
* test(e2e): settle the projects fetch in real-daemon-run home navigation
With serial mode gone, the plugin-authoring spec runs after arbitrary
shard predecessors, so the daemon holds more projects when it navigates
home; the late-resolving projects list re-renders the hero and left the
shortcuts trigger unstable for the whole click timeout on CI. Port the
projects-settle waiter already used by entry-chrome-flows into this
file's gotoEntryHome.
* test(e2e): name the plugins-load gate in the plugin authoring spec
The home shortcuts trigger is disabled while the plugins list loads, and
this spec's mid-flow reload re-fires that fetch after gotoEntryHome's
settle has already passed — on CI the click then times out opaquely for
120 seconds. Arm a response waiter across the reload and assert the
trigger's enabled state explicitly, so a slow or hung /api/plugins fails
with the actual cause named.
* test(e2e): enter plugin authoring through the plugins page create button
The home shortcuts trigger sits disabled on CI runners while a home
plugin apply hangs (plugins list resolves, the pending-apply gate never
clears); three reproduction attempts confirmed the anomaly is CI-only
and independent of this journey. The create button on the Plugins page
drives the same queuePluginAuthoring flow, and this spec's oracle is the
generated scaffold plus its action cards, so route the entry there and
track the disabled-trigger anomaly separately.
* ci: let configure-ci-parallelism drive full UI shard workers
The single-worker override predates the repairs that explained the
two-worker instability (BYOK agent availability and the reveal remount
race, both test-side and since fixed), so it was pinning throughput on a
stale diagnosis. Drop it and let the repo's nproc/2 convention apply —
two workers on the four-core ui_hot profile — halving the per-shard test
step if the runtime holds.
* ci: probe three full UI workers against Blacksmith's actual vCPU capacity
The ui_hot profile resolves to blacksmith-4vcpu runners that reportedly
provide six actual vCPUs while nproc's cgroup view says four, so the
nproc/2 convention derives two workers and idles capacity. Pin three
workers for this probe and print the CPU topology alongside for
verification.
* Revert "ci: probe three full UI workers against Blacksmith's actual vCPU capacity"
This reverts commit 63c8296.
* ci: skip the critical fallback for pure packaged-leaf changes
Playwright starts tools-dev web (daemon + web runtimes) and never
launches the desktop, packaged, or tools-pack entrypoints, so PRs
confined to apps/desktop, apps/packaged, or tools/pack were paying the
two-job critical fallback (5.4-13.4 minutes of wall each) for coverage
their package tests and preflight already provide — seven of the eight
fallback PRs in the latest 100-merge sample were such false positives.
Add a fail-closed ui_critical_validation_required scope: it clears only
when every non-exempt changed file sits under one of those three leaf
roots. tools-dev, any transitive package (including the undeclared
metatool edge), scripts, runtime resources, unknown roots, and mixed
leaf + runtime changes keep the fallback armed; root manifests keep
enabling P0 with the existing critical/P0 mutual exclusion; merge-group
full mode is unchanged.
* test(e2e): align question-form specs with inline Studio discovery
Rebasing onto main picked up #5496, which moved clarification forms
inline into the chat flow and removed the legacy Questions workspace
tab. Re-point the submit-persistence spec at the inline form, and fix
the oracles that no other lane had verified since: the answered summary
echoes the picked visual-style card title (not the option label), and
the submitted answer message carries the card title plus its value id.
* test(e2e): absorb post-rebase drift in BYOK gating and zh-CN hero copy
Two more contracts only the full pool guards moved under the rebase:
- The composer's BYOK gate silently swallows submits until the
incremental /api/agents stream has delivered byok-opencode's
availability, so a fast submit races the stream (the sendPrompt
diagnostic reports the POST was never issued). Retry through that
window; runErrorCard reads the last card, so swallowed attempts cannot
shadow the provider-validation oracle. The gate erroring instead of
waiting while agents are still loading is a product-side follow-up.
- The home hero title copy changed in every locale; re-point the zh-CN
P2 smoke at the current translation.
* test(e2e): align gallery specs with the all-types default slice
The Community gallery now defaults to an All category pill (#5759)
instead of pre-selecting Deck: assert the new default state in the hero
and category-tab specs, and have the facet-filter spec pick Deck
explicitly before asserting the deck-only visibility set.
* test(e2e): align the first-run reveal spec with the all-types default slice
Same #5759 drift family as the entry-chrome-flows gallery specs, missed
by the first sweep because it lives in home-hero-rail: the revealed
gallery now shows the All pill selected instead of hiding it with
live-artifact pre-selected.
* docs(e2e): add UI test stability rules distilled from the full-pool work
Capture the invariants a UI test must hold to stay green under the generic
sharded full pool, learned from stabilizing that suite and two rebases:
- order independence (no serial groups; setup is per-file, not universal),
and that neither shade-filtered local runs nor merge-path P0 prove it;
- retry-only passes are an order-leak/readiness signal, not flake;
- settle async surfaces (best-effort waiter + a state-survives-a-window
check) before interacting;
- an enabled control is not a ready control when a streamed precondition
(e.g. BYOK-OpenCode agent availability over the /api/agents SSE stream)
gates it;
- never force-click into inert/collapsed reveal containers; use
actionability, not visibility, as the readiness signal;
- hermetic agent availability via route mocks (with the terminal SSE `done`
event) or fake CLI runtimes + agentCliEnv, never host binaries;
- realign stale oracles against the product and confirm them in the lane
that actually executes the case, since a merged PR does not verify P1/P2
or unregistered-file P0 oracles;
- name failure causes so a CI-only reproduction carries its own evidence.
Cross-linked from the top-of-file coverage-status pointer.
* test(e2e): realign BYOK missing-provider spec to the #5745 client-side preflight
main #5745 added a client-side BYOK preflight that blocks a missing-provider
run before any POST and opens the execution Settings section, instead of
letting the run reach the daemon and fail there. Rewrite the spec to the new
"blocked before spawn" behavior: submitting opens the execution-mode dialog,
no create-run request is issued, and no artifact is produced (persisted
across reload). This drops the now-obsolete daemon-failed-run assertions and
the SSE-availability retry loop, which the preflight makes unnecessary.
* docs(e2e): explain why the functional config ignores visual specs
Clarify that visual-*.test.ts are owned by playwright.visual.config.ts and
its playwright_visual lane, so excluding them from the functional config is
intentional scope (it keeps a bare test:ui or the full-pool shard run from
executing visual specs without their visual config), not a disabled lane.
* fix(web): portal the New Project brand-preview flyout with the popover
The earlier DS-picker unclip portaled the popover list but left its companion
brand-preview flyout on the old inline `position: absolute` against the
trigger wrapper. Once the popover moved to a body portal, the flyout for a
finalized `user:<id>` brand was still clipped by the modal body and no longer
anchored to the popover. Portal the flyout too and place it beside the
popover with the same viewport-aware logic (right of it when the 320px card
fits, else flipped left, then clamped), mirroring the popover's up/down
vertical anchor.
Red spec: e2e/ui/new-project-ds-picker-clipping.test.ts
"finalized-brand preview flyout is not clipped by the modal body" — red with
the popover portaled but the flyout inline, green with this change.1 parent 30a4c28 commit 20c6c0a
35 files changed
Lines changed: 1195 additions & 611 deletions
File tree
- .github/workflows
- apps/web
- src
- components
- styles/workspace
- tests/components
- e2e
- lib/playwright
- tests
- ui
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
441 | 441 | | |
442 | 442 | | |
443 | 443 | | |
444 | | - | |
445 | | - | |
| 444 | + | |
| 445 | + | |
446 | 446 | | |
447 | 447 | | |
448 | 448 | | |
449 | | - | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
450 | 454 | | |
451 | 455 | | |
452 | 456 | | |
| |||
476 | 480 | | |
477 | 481 | | |
478 | 482 | | |
479 | | - | |
480 | | - | |
| 483 | + | |
| 484 | + | |
481 | 485 | | |
482 | | - | |
483 | | - | |
| 486 | + | |
| 487 | + | |
484 | 488 | | |
485 | 489 | | |
486 | | - | |
| 490 | + | |
487 | 491 | | |
488 | 492 | | |
489 | 493 | | |
| |||
492 | 496 | | |
493 | 497 | | |
494 | 498 | | |
495 | | - | |
496 | | - | |
497 | | - | |
498 | | - | |
499 | | - | |
500 | | - | |
501 | | - | |
502 | | - | |
503 | | - | |
504 | | - | |
505 | | - | |
506 | | - | |
507 | | - | |
508 | | - | |
509 | | - | |
510 | | - | |
511 | | - | |
512 | | - | |
513 | | - | |
514 | | - | |
515 | | - | |
516 | | - | |
517 | | - | |
518 | | - | |
519 | | - | |
520 | | - | |
521 | | - | |
522 | | - | |
523 | | - | |
524 | | - | |
525 | | - | |
526 | | - | |
527 | | - | |
528 | | - | |
529 | | - | |
530 | | - | |
531 | | - | |
532 | | - | |
533 | | - | |
534 | | - | |
535 | | - | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
536 | 502 | | |
537 | 503 | | |
538 | 504 | | |
| |||
648 | 614 | | |
649 | 615 | | |
650 | 616 | | |
651 | | - | |
652 | 617 | | |
653 | 618 | | |
654 | 619 | | |
| |||
695 | 660 | | |
696 | 661 | | |
697 | 662 | | |
698 | | - | |
| 663 | + | |
699 | 664 | | |
700 | 665 | | |
701 | 666 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
5 | | - | |
6 | | - | |
7 | 4 | | |
8 | 5 | | |
9 | 6 | | |
| |||
24 | 21 | | |
25 | 22 | | |
26 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
27 | 41 | | |
28 | 42 | | |
| 43 | + | |
29 | 44 | | |
30 | | - | |
| 45 | + | |
31 | 46 | | |
32 | 47 | | |
33 | 48 | | |
34 | 49 | | |
35 | 50 | | |
36 | 51 | | |
37 | 52 | | |
| 53 | + | |
| 54 | + | |
38 | 55 | | |
39 | 56 | | |
40 | 57 | | |
| |||
44 | 61 | | |
45 | 62 | | |
46 | 63 | | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
47 | 67 | | |
48 | 68 | | |
| 69 | + | |
| 70 | + | |
49 | 71 | | |
50 | 72 | | |
51 | 73 | | |
52 | 74 | | |
53 | 75 | | |
54 | 76 | | |
| 77 | + | |
55 | 78 | | |
56 | 79 | | |
57 | 80 | | |
| |||
62 | 85 | | |
63 | 86 | | |
64 | 87 | | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | 88 | | |
69 | 89 | | |
70 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
71 | 108 | | |
72 | 109 | | |
73 | 110 | | |
74 | 111 | | |
75 | | - | |
| 112 | + | |
76 | 113 | | |
77 | 114 | | |
78 | 115 | | |
| |||
92 | 129 | | |
93 | 130 | | |
94 | 131 | | |
95 | | - | |
| 132 | + | |
96 | 133 | | |
97 | 134 | | |
98 | | - | |
| 135 | + | |
99 | 136 | | |
100 | 137 | | |
101 | 138 | | |
| |||
136 | 173 | | |
137 | 174 | | |
138 | 175 | | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
143 | 185 | | |
144 | 186 | | |
145 | 187 | | |
146 | 188 | | |
147 | 189 | | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
148 | 193 | | |
149 | 194 | | |
| 195 | + | |
| 196 | + | |
150 | 197 | | |
151 | 198 | | |
152 | 199 | | |
153 | 200 | | |
154 | 201 | | |
155 | 202 | | |
| 203 | + | |
156 | 204 | | |
157 | 205 | | |
158 | 206 | | |
| |||
163 | 211 | | |
164 | 212 | | |
165 | 213 | | |
166 | | - | |
167 | | - | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
168 | 221 | | |
169 | | - | |
| 222 | + | |
170 | 223 | | |
171 | 224 | | |
172 | 225 | | |
173 | | - | |
| 226 | + | |
174 | 227 | | |
175 | 228 | | |
176 | 229 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
260 | 260 | | |
261 | 261 | | |
262 | 262 | | |
263 | | - | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
264 | 269 | | |
265 | 270 | | |
266 | 271 | | |
| |||
1338 | 1343 | | |
1339 | 1344 | | |
1340 | 1345 | | |
1341 | | - | |
| 1346 | + | |
1342 | 1347 | | |
1343 | 1348 | | |
1344 | 1349 | | |
| |||
1668 | 1673 | | |
1669 | 1674 | | |
1670 | 1675 | | |
1671 | | - | |
| 1676 | + | |
1672 | 1677 | | |
1673 | 1678 | | |
1674 | 1679 | | |
| |||
2112 | 2117 | | |
2113 | 2118 | | |
2114 | 2119 | | |
2115 | | - | |
| 2120 | + | |
2116 | 2121 | | |
2117 | 2122 | | |
2118 | 2123 | | |
| |||
2141 | 2146 | | |
2142 | 2147 | | |
2143 | 2148 | | |
2144 | | - | |
| 2149 | + | |
2145 | 2150 | | |
2146 | 2151 | | |
2147 | 2152 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
632 | 632 | | |
633 | 633 | | |
634 | 634 | | |
635 | | - | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
636 | 638 | | |
637 | 639 | | |
638 | 640 | | |
| |||
0 commit comments