|
| 1 | +# Clear and responsive live comparison |
| 2 | + |
| 3 | +## Traceability |
| 4 | + |
| 5 | +- Spec ID: studio-compare-ux-refinement |
| 6 | +- Status: Implemented |
| 7 | + |
| 8 | +## Intent |
| 9 | + |
| 10 | +Make the default live Compare surface answer three user questions in order: |
| 11 | +which project and Agents will run, what variable the run can honestly compare, |
| 12 | +and what observable differences the two runs produced. Preserve the existing |
| 13 | +resource-linked Tool Call evidence while making the setup and results usable at |
| 14 | +390px without clipped controls or character-wide text. |
| 15 | + |
| 16 | +## Acceptance Scenarios |
| 17 | + |
| 18 | +- AC-1: At 390x844, Current project, both Agent selectors, Prompt, comparison |
| 19 | + scope, Advanced details, and Run compare remain fully inside the viewport, |
| 20 | + readable, and keyboard reachable. No paragraph or textarea collapses to a |
| 21 | + character-wide column. |
| 22 | +- AC-2: Before Run, the surface names the selected Agent and effective model |
| 23 | + policy for each lane and states the observable comparison variables. Two |
| 24 | + effectively identical selections are labelled as a repeatability comparison; |
| 25 | + multiple changed variables are labelled descriptive rather than controlled. |
| 26 | +- AC-3: A single checkpoint-bound project is presented as fixed context rather |
| 27 | + than as a one-option selector. The UI does not claim arbitrary project |
| 28 | + rebinding. |
| 29 | +- AC-4: Run compare is the only visually primary action in the composer. |
| 30 | + Advanced details remains available as a lower-weight disclosure, and status |
| 31 | + copy explains readiness instead of saying only `Ready`. |
| 32 | +- AC-5: Lane headers lead with the selected Agent, then show the effective model |
| 33 | + separately. `Agent default` is never presented as the Agent's primary name. |
| 34 | +- AC-6: Completed results lead with a factual delta summary derived only from |
| 35 | + retained lane state and projected operations: lane outcome, shared/run-only |
| 36 | + resources, edits, and verification activity. It does not emit an aggregate |
| 37 | + quality score or single-trial winner. |
| 38 | +- AC-7: Wide and compact resource comparison retains the symmetric three-column |
| 39 | + map. Narrow comparison renders each resource as one row header followed by |
| 40 | + full-width AI 1 and AI 2 operation sections, keeping inline Tool Call results |
| 41 | + owned by their originating lane. |
| 42 | +- AC-8: Browser regression checks assert bounding rectangles for critical |
| 43 | + controls, meaningful minimum Prompt width, primary-action visibility, |
| 44 | + keyboard tab semantics, no document-level overflow, and screenshots at |
| 45 | + 1440x900, 1024x768, and 390x844 with no console/page errors. |
| 46 | + |
| 47 | +## Non-goals |
| 48 | + |
| 49 | +- Add filesystem project discovery or implement the workspace chooser in this |
| 50 | + change. |
| 51 | +- Rank an Agent or model from one comparison, add a composite score, or infer |
| 52 | + hidden intent from Tool Calls. |
| 53 | +- Replace Messages or Advanced evidence, redesign Evidence results, or change |
| 54 | + experiment execution and ACP permission semantics. |
| 55 | +- Add new Agent adapters or installation actions. |
| 56 | + |
| 57 | +## Plan and Tasks |
| 58 | + |
| 59 | +1. Add pure comparison-scope and factual-result projections beside the existing |
| 60 | + resource comparison model, with focused unit coverage. |
| 61 | +2. Refactor the Simple Compare composer into fixed project context, explicit |
| 62 | + Agent/model lanes, one comparison-scope row, and one primary Run action. |
| 63 | +3. Add the factual result summary above Resources/Messages and correct lane |
| 64 | + header hierarchy. |
| 65 | +4. Override every Agent-catalog grid placement at the narrow breakpoint and |
| 66 | + stack each resource's two lane sections vertically. |
| 67 | +5. Strengthen Playwright assertions around actual element bounds and retain |
| 68 | + screenshots for wide, compact, and narrow review. |
| 69 | + |
| 70 | +## Test and Review Evidence |
| 71 | + |
| 72 | +- AC-2/AC-6: comparison-model tests cover identical Agent-default lanes, |
| 73 | + lane-model changes, mixed Agent/model-policy attribution, shared/run-only |
| 74 | + resources, edits, verification, and failed lane state. |
| 75 | +- AC-1/AC-3/AC-4/AC-5/AC-7/AC-8: built Studio Playwright checks inspect control |
| 76 | + bounds, Prompt width, action visibility, headers, resource stacking, tab |
| 77 | + keyboard behavior, overflow, screenshots, and console/page errors at all |
| 78 | + three design widths. |
| 79 | +- Validation target: `npm run harness-studio:test`, the complete Studio browser |
| 80 | + spec, `npm run check`, documentation link-graph validation, and |
| 81 | + `git diff --check` under Node 24. |
| 82 | +- Risk: a concise scope label can still overstate attribution. Derive it only |
| 83 | + from selected Agent ids, model policies, and lane models; call any multi-axis |
| 84 | + movement descriptive. |
| 85 | +- Risk: stacking lanes can obscure cross-lane alignment. Keep a single resource |
| 86 | + header and stable AI 1 then AI 2 ordering, with full-width operation controls. |
| 87 | +- Risk: summary copy can turn observations into a verdict. Use factual verbs |
| 88 | + such as observed, edited, verified, passed, failed, or cancelled and retain |
| 89 | + Advanced evidence for sufficiency and controlled attribution. |
| 90 | + |
| 91 | +## Implementation Evidence |
| 92 | + |
| 93 | +- `PATH="/opt/homebrew/opt/node@24/bin:$PATH" npm run check` passed: 1,545 |
| 94 | + root tests with 2 skips, 172 Harness tests, 31 Harness UI tests, 293 Studio |
| 95 | + tests, generated-source checks, and package verification. |
| 96 | +- `npx playwright test packages/harness-studio/test/browser/tool-call.spec.mjs` |
| 97 | + passed all 10 browser scenarios with empty console/page-error collections. |
| 98 | +- Wide, compact, and narrow `bench-*.png` and |
| 99 | + `compare-resource-map-*.png` screenshots were inspected for hierarchy, |
| 100 | + clipping, overflow, and resource-to-lane association. |
| 101 | +- `npx vitest run test/skills-docs/doc-link-graph.test.mjs` passed all 8 link |
| 102 | + graph checks after regenerating `docs/better-harness-doc-links.mmd`; the |
| 103 | + graph remained unchanged. |
| 104 | +- `git diff --check` passed, and the change adds no generated or package |
| 105 | + artifacts to the worktree. |
0 commit comments