Mobile-responsive Agent UI: touch targets, single-pane layout, virtual keyboard

[kovtcharov]

Goal

Audit and fix the Agent UI for actual mobile usability. The desktop UI shrunk to a phone-sized viewport is not a mobile experience — touch targets, single-pane layouts, virtual keyboard handling all need explicit attention.

Why this matters for consumer adoption

The tunnel (#872) gives mobile reach. The PWA (M1) gives mobile install. But if the UI itself feels like a desktop app crammed onto a phone, neither matters. Mobile-first audit is the third leg of the tripod.

Scope (1–2 PRs, v0.18.2)

A. Mobile UX audit

• Document every Agent UI screen at 375×667 (iPhone SE — narrowest viewport we care about), 390×844 (iPhone 14), 412×915 (Pixel 7)
• Catalog every issue: cut-off text, overlapping buttons, hidden scrollbars, sub-44pt touch targets, fixed widths
• Output: docs/spec/agent-ui-mobile-audit.md with screenshots + per-screen issue list

B. Layout fixes — single-pane mobile collapse

• Three-pane desktop layout (sidebar / chat / detail) collapses to single-pane stack on mobile with bottom-tab navigation OR drawer pattern
• Sidebar (sessions, agents, MCP) becomes a slide-out drawer triggered by hamburger
• Settings / Memory Dashboard / MCP panel become full-screen modals on mobile (not floating panels)
• Chat input sticks to bottom; account for iOS safe-area-inset

C. Touch + keyboard handling

• All interactive elements ≥ 44×44pt (Apple HIG minimum)
• No hover-only interactions — everything must work on touch
• Virtual keyboard handling: viewport adjusts; chat input stays visible above keyboard; scroll jumps fixed
• iOS scroll bounce / over-scroll behaves correctly inside chat history

D. Mobile-specific affordances

• Pull-to-refresh on chat history (native expectation on mobile)
• Long-press on a message → context menu (copy, edit, delete)
• File upload via mobile photo picker / file picker
• Camera capture for image input (already in scope for v0.21.0 multimodal but should work via mobile browser)

E. Tests

• Playwright (Agent UI: Playwright + Strix Halo E2E suite (validate every UI feature on real AMD hardware) #883) mobile viewport coverage — add iPhone SE, Pixel 7 device profiles to the test matrix
• Screenshot snapshots at 3 mobile viewports for regression detection
• Lighthouse mobile audit score ≥ 90 on chat page

What this is NOT

• ❌ Not a complete UI redesign — keep the existing visual language; fix the layout
• ❌ Not native iOS / Android apps
• ❌ Not the PWA shell (M1)
• ❌ Not push notifications (M4)

Acceptance criteria

• Every Agent UI screen is usable at 375×667 with no horizontal scroll, no cut-off interactive elements
• All touch targets meet 44pt minimum
• Virtual keyboard does not obscure chat input on iOS Safari + Android Chrome
• Lighthouse mobile audit ≥ 90 on chat page
• Playwright mobile-viewport tests in Agent UI: Playwright + Strix Halo E2E suite (validate every UI feature on real AMD hardware) #883 pass

Attribution / prior art

• Apple HIG — touch target minimums, safe-area-inset handling
• Material Design — bottom navigation patterns

Dependencies

• Adjacent: feat(ui): friendly ngrok tunnel diagnostics + cookie auth for mobile #872 (tunnel — primary mobile entry path), M1 (PWA shell), Agent UI: Playwright + Strix Halo E2E suite (validate every UI feature on real AMD hardware) #883 (Playwright E2E mobile coverage)
• Adjacent: existing Agent UI work in src/gaia/apps/webui/

[kovtcharov]

Spec — Issue #895: Mobile-responsive Agent UI (M2)

Depends on: mobile_design_system.md (the
design-system doc is the foundation; this spec is the implementation
of it inside src/gaia/apps/webui/).

Blocks: #896 (mobile voice — needs the chat input row redesigned
first), #898 (PWA — needs viewport/safe-area landed).

Touches: src/gaia/apps/webui/src/ (frontend only).

---

1. Goal

After M1 (#893), a phone can reach the Agent UI through a tunnel.
After M2 (#895), a phone can use it: the sidebar drawer works,
modals are full-screen, the chat input row sits above the keyboard,
and every interactive element is at least 44×44.

Success = a first-time mobile user can create a session, send a
message, view a streamed response, attach a document, change settings,
and view the document library — all without zooming, mis-tapping, or
losing the input behind the keyboard.

---

2. Component changes (mapped to existing code)

2.1 New directory: src/gaia/apps/webui/src/components/mobile/

Per design-system doc §10. Create:

• Panel.tsx + Panel.css — replaces inline modal styles
• BottomSheet.tsx + .css — for action sheets, confirmations
• MobileHeader.tsx + .css — hamburger + title slot
• TouchTarget.tsx + .css — enforces 44×44 + aria-label
• index.ts — barrel export

VoiceButton.tsx is created in #896, not here.

2.2 New utilities

• src/gaia/apps/webui/src/utils/breakpoints.ts — MOBILE_MAX_PX,
  isMobileViewport(). Replaces five inline window.innerWidth <= 768
  checks in App.tsx (lines 309–317, 332, 372, 449, 452).
• src/gaia/apps/webui/src/hooks/useViewportHeight.ts — Visual
  Viewport API wrapper from design-system doc §8.
• src/gaia/apps/webui/src/hooks/useIsMobile.ts — replaces the
  useState/useEffect resize block in App.tsx:447–455.

2.3 Edits to existing files

File	Change
index.html	Add viewport-fit=cover to <meta name="viewport">
styles/index.css	Add --bp-mobile-max, --safe-* tokens, focus-ring tweak
App.tsx	Replace inline 768 checks; wrap modals in <Panel>
components/Sidebar.tsx + .css	44×44 hit area on toggle, drawer focus management
components/ChatView.tsx + .css	Sticky input row using useViewportHeight(); bigger Send
components/MessageBubble.tsx	Tap-to-reveal action menu on touch (replaces hover-only)
components/SettingsModal.tsx	Refactor to use <Panel>
components/DocumentLibrary.tsx	Refactor to use <Panel wide>
components/FileBrowser.tsx	Refactor to use <Panel wide>

---

3. Layout collapse logic

Single rule, single source of truth: on
isMobileViewport(), all <Panel> instances render full-screen
(position: fixed; inset: 0) and slide up from bottom; off-mobile
they render as the existing right-side floating cards.

Decision tree at render time:

mobile?
├─ yes → Panel = full-screen, slide-up, focus trap, body inert
└─ no  → Panel = floating right-side card (current desktop behavior)

sidebar?
├─ mobile → drawer overlay, default closed, backdrop tap closes
└─ desktop → persistent column, default open, no backdrop

No "tablet" branch — 769–1023px renders the desktop layout. If body
content would clip below 900px, the sidebar collapses to icons-only;
this is already the existing behavior and is not part of M2.

---

4. Touch-target audit + fix list

Per design-system doc §6. Concrete fixes for M2:

Component	File	Current	Fix
Hamburger toggle	App.tsx:481-487	<button> with 18px icon, no padding	Wrap in <TouchTarget> (44×44)
Session row	Sidebar.tsx (SessionItem)	full-row click ok; trash icon small	Trash icon → <TouchTarget size={44}>; always visible on touch
Sidebar settings/theme/sun/moon	Sidebar.tsx bottom row	tightly packed	8px min gap; each in <TouchTarget>
Chat input Send	ChatView.tsx	TBD	44×44 primary CTA, --amd-red fill
Chat input attach (Paperclip)	ChatView.tsx	small icon button	<TouchTarget>
Chat input edit (Edit3) / others	ChatView.tsx	small	<TouchTarget>
MessageBubble actions (copy/regen)	MessageBubble.tsx	hover-only, small	tap-to-reveal <BottomSheet>
ConnectionBanner dismiss	ConnectionBanner.tsx	TBD	<TouchTarget>
Toast (create error)	App.tsx:550	not interactive	leave; auto-dismiss already

Tracking: each row above is one PR-sized chunk; M2 may ship as a
stack of small PRs rather than one mega-PR.

---

5. Tests

Uses the Playwright harness from #883. New file:
tests/electron/mobile-responsive.spec.ts (or wherever #883 lands).

5.1 Viewport profiles (run all tests against each)

const MOBILE_PROFILES = [
  { name: 'iPhone 13', viewport: { width: 390, height: 844 }, hasTouch: true },
  { name: 'Pixel 7',   viewport: { width: 412, height: 915 }, hasTouch: true },
  { name: 'iPad mini', viewport: { width: 768, height: 1024 }, hasTouch: true },
  { name: 'desktop',   viewport: { width: 1440, height: 900 }, hasTouch: false },
];

768 deliberately straddles the boundary — on iPad mini portrait the
layout should resolve to desktop (because the rule is ≤ 768,
and 768 itself is desktop... wait — re-read: design-system doc §3 says
mobile is ≤ 768, so 768 is mobile; iPad mini portrait is 768 wide
so it gets the mobile layout). Test asserts the chosen branch
explicitly so the rule is documented in the test.

5.2 Required test cases

1. Drawer behavior: opens on hamburger, closes on backdrop tap,
   closes on Esc, returns focus to hamburger.
2. Modal collapse: opening Settings on mobile renders full-screen;
   on desktop renders right-side card.
3. Touch targets: for every interactive element on every viewport,
   boundingBox().width >= 44 && height >= 44. Loop, don't enumerate.
4. Input above keyboard: simulate keyboard with
   page.evaluate(() => window.visualViewport) mock; assert input row
   getBoundingClientRect().bottom <= visualViewport.height.
5. Safe-area: with viewport-fit=cover, assert no content is
   placed under env(safe-area-inset-top/bottom) zones (use a
   CSS-injected sentinel).
6. Streaming a11y: start a chat, assert the streaming message has
   aria-live="polite".
7. axe-core scan: zero AA violations on every viewport on every
   primary screen (welcome, chat, settings, documents).
8. Reduced motion: with page.emulateMedia({ reducedMotion: 'reduce' }),
   assert no transition-duration > 100ms on enter/exit animations.

5.3 Manual QA

A separate one-page checklist at docs/qa/mobile-m2-checklist.md
(create as part of this issue) covers the things Playwright can't do
easily — real device testing on iOS Safari + Android Chrome through
the M1 tunnel.

---

6. Acceptance criteria

• components/mobile/ exists with Panel, BottomSheet,
  MobileHeader, TouchTarget and per-component CSS.
• All five inline window.innerWidth <= 768 checks in App.tsx
  replaced by isMobileViewport().
• index.html viewport tag includes viewport-fit=cover.
• Settings, DocumentLibrary, FileBrowser use <Panel>.
• Hamburger, session-row trash, all chat input icon buttons,
  all sidebar bottom-row icons go through <TouchTarget>.
• Streaming message has aria-live="polite".
• All Playwright tests in §5.2 pass on all four profiles.
• axe-core CI gate passes with zero AA violations.
• Manual QA checklist passes on iOS Safari (16.4+) and Android
  Chrome (latest stable) through M1 tunnel.

---

7. Failure modes & mitigations

Failure	Likelihood	Mitigation
iOS Safari ignores visualViewport resize on first focus	medium	Hook polls + listens; tested on real device, not just emulator
Existing modal animations conflict with new <Panel> slide-up	high	Migrate one modal at a time; keep old AnimatedPresence wrapper, add new slide-up exit class scoped to mobile
Trash icon "always visible on touch" breaks desktop hover-to-reveal aesthetic	low	Use @media (hover: none) to gate the always-visible state — desktop with mouse keeps hover-only
Drawer + body inert breaks screen reader announcements during streaming	medium	Place the streaming live region outside the body container that gets inert, or move it into the drawer-open conditional
44×44 audit creates visual regressions on desktop	low	<TouchTarget> only enforces minimum hit-area via padding; visual size of icon stays the same
Migrating five inline 768 checks to isMobileViewport() introduces a regression in sidebar auto-close	medium	Add a focused unit test on isMobileViewport and a Playwright test that creates a session at 390px and asserts sidebar closes

---

8. Out of scope (track separately)

• Voice button — Voice via mobile browser: ASR + TTS on iOS Safari + Android Chrome #896 (M3)
• PWA manifest, install prompt, offline cache — Tunnel share UX: QR code + Add-to-Home-Screen + URL persistence #898 (M5)
• Tablet-specific layout — defer until telemetry justifies
• Right-to-left language support — v0.21 backlog
• Swipe-from-edge to open sidebar — P2 enhancement, post-M2

---

9. References

• Design system: mobile_design_system.md
• App entry: src/gaia/apps/webui/src/App.tsx
• Existing styles: src/gaia/apps/webui/src/styles/index.css
• Sidebar: src/gaia/apps/webui/src/components/Sidebar.tsx
• ChatView: src/gaia/apps/webui/src/components/ChatView.tsx
• Playwright harness: Issue Agent UI: Playwright + Strix Halo E2E suite (validate every UI feature on real AMD hardware) #883