Agent UI: multi-agent task management platform with per-task windows and full observability

[kovtcharov]

Problem

The current Agent UI is a single-agent chat interface — one conversation, one agent, one window. The GaiaAgent architecture (#674) introduces multiple agents working on multiple tasks simultaneously. The UI needs to become a management platform where users can observe and interact with all agents and their tasks.

Vision

The Agent UI becomes the control plane for a multi-agent system:

• Users see all active tasks across all agents
• Each task has its own conversation window (like browser tabs)
• Any agent or the user can participate in a task's conversation
• Everything is observable — tool calls, inter-agent messages, memory writes, task status

UI Architecture

Task-Centric Layout

┌─────────────────────────────────────────────────────┐
│  GAIA Agent UI                              [+ New]  │
├───────────┬─────────────────────────────────────────┤
│           │                                         │
│  TASKS    │   ACTIVE TASK WINDOW                    │
│           │                                         │
│  ● Chat   │   [GaiaAgent] Hey! What can I help      │
│    with    │   you with today?                       │
│    Kalin   │                                         │
│           │   [You] Summarize my Q3 report and      │
│  ◐ Q3     │   check my calendar for tomorrow        │
│    Report  │                                         │
│    Summary │   [GaiaAgent] On it! I've started two   │
│    (RAG)   │   tasks for you:                        │
│           │   📄 Q3 Report Summary → DocAgent        │
│  ◐ Check  │   📅 Tomorrow's Calendar → CalendarAgent │
│    Calendar│                                         │
│    (Cal)   │   ──── SUB-TASK: Q3 Report ────         │
│           │   [DocAgent] Indexing Q3_report.pdf...   │
│  ✓ Email  │   [DocAgent→FileAgent] Need file at     │
│    Digest  │   ~/docs/Q3_report.pdf                  │
│    (Email) │   [FileAgent] Found. 45 pages, 2.1MB.  │
│           │   [DocAgent] Indexed. Key findings:      │
│  AGENTS   │   • Revenue: $4.2M (+12% QoQ)           │
│  ● Gaia   │   • ...                                  │
│  ● RAG    │                                         │
│  ○ File   │   ──── SUB-TASK: Calendar ────           │
│  ○ Shell  │   [CalendarAgent] Tomorrow's agenda:     │
│  ○ Web    │   • 9am: Team standup                    │
│  ○ Email  │   • 2pm: Q3 review (related!)           │
│  ○ Calendar│                                         │
│           │   [GaiaAgent] Here's the summary...      │
├───────────┴─────────────────────────────────────────┤
│  [Type a message...]                        [Send]   │
└─────────────────────────────────────────────────────┘

Key UI Components

1. Task Sidebar

• List of all active/completed tasks
• Status indicators: ● active, ◐ in progress, ✓ completed, ✗ failed
• Which agent is assigned to each task
• Click to switch task window (like browser tabs)
• Nested sub-tasks visible under parent tasks

2. Per-Task Conversation Window

• Each task has its own conversation thread
• Multiple participants: user + GaiaAgent + specialist agents
• Messages labeled with sender: [GaiaAgent], [DocAgent], [You]
• Inter-agent messages visible (DocAgent → FileAgent) for full observability
• Tool calls shown inline with expandable details

3. Agent Status Panel

• List of all registered agents with status (active/idle/unavailable)
• Current task assignment per agent
• Memory stats: insights count, conversation turns, tool calls
• Model info: which model, which device (NPU/GPU/CPU)

4. Observability Features

• Tool trace view: Every tool call with args, result, duration
• Inter-agent messages: Who asked who for what
• Memory writes: When an agent stores a new insight, show it
• Task timeline: Gantt-like view of task execution with agent handoffs
• Cost tracking: Tokens used per agent per task

Interaction Model

User can:

• Chat with GaiaAgent in the main window (like today)
• Click into any sub-task to see its conversation
• Send messages within any task's conversation (not just GaiaAgent's)
• Create tasks manually ("Hey DocAgent, index this folder")
• Pause/cancel/retry tasks

GaiaAgent can:

• Create tasks (via MCP Agent MCP Server: task creation, agent assignment, and inter-agent communication #675) — they appear in sidebar instantly
• Assign tasks to specialists — status updates in real-time
• Report back to user in main conversation

Specialist agents can:

• Create sub-tasks for other agents (all-to-all via MCP Agent MCP Server: task creation, agent assignment, and inter-agent communication #675)
• Post updates to their task conversation
• Complete tasks with results
• All activity visible in the task window

Backend Requirements

Extend src/gaia/ui/ with:

# New router: src/gaia/ui/routers/tasks.py
POST   /api/tasks                    # Create task
GET    /api/tasks                    # List tasks (with filters)
GET    /api/tasks/{id}               # Get task with messages
PATCH  /api/tasks/{id}               # Update status
DELETE /api/tasks/{id}               # Cancel task
POST   /api/tasks/{id}/messages      # Add message to task conversation
GET    /api/tasks/{id}/timeline      # Get task execution timeline
GET    /api/tasks/{id}/traces        # Get tool call traces

# New router: src/gaia/ui/routers/agents.py  
GET    /api/agents                   # List all agents with status
GET    /api/agents/{id}              # Agent details + capabilities
GET    /api/agents/{id}/memory       # Agent's memory (insights, stats)
GET    /api/agents/{id}/tasks        # Agent's active/completed tasks

New SSE events:

task_created      — new task appears in sidebar
task_updated      — status change (pending → in_progress → completed)
task_message      — new message in a task conversation  
agent_status      — agent went active/idle/unavailable
agent_insight     — agent stored a new insight (observable)
inter_agent_msg   — agent-to-agent message (for observability)

Frontend Components (new)

src/gaia/apps/webui/src/components/
├── TaskSidebar.tsx          — Task list with status/nesting
├── TaskWindow.tsx           — Per-task conversation view
├── TaskTimeline.tsx         — Gantt-like execution timeline
├── AgentStatusPanel.tsx     — Agent list with status/memory
├── InterAgentTrace.tsx      — Agent-to-agent message visualization
├── ToolTraceView.tsx        — Expandable tool call details
└── TaskCreator.tsx          — Manual task creation dialog

How This Connects

• Agent decomposition: GaiaAgent (0.6B) orchestrates specialized sub-agents for tool use #674 GaiaAgent — Creates tasks that appear in sidebar
• Agent MCP Server: task creation, agent assignment, and inter-agent communication #675 Agent MCP Server — All task/agent operations flow through MCP
• Shared memory database with per-agent namespaces for multi-agent architecture #676 Shared Memory — Agent insights visible in Agent Status Panel
• Agent UI: Agent Registry and Selection #612 Agent Registry — Powers the Agent Status Panel

Dependencies

• Agent MCP Server: task creation, agent assignment, and inter-agent communication #675 — Agent MCP Server (provides task/agent APIs)
• Shared memory database with per-agent namespaces for multi-agent architecture #676 — Shared memory (provides agent insight visibility)
• Agent UI: Agent Registry and Selection #612 — Agent Registry (provides agent discovery)
• Current Agent UI (extend, don't rewrite)

Acceptance Criteria

• Tasks appear in sidebar with real-time status updates via SSE
• Each task has its own conversation window with multi-participant messages
• User can click between tasks (tab-like navigation)
• Inter-agent messages visible within task conversations
• Tool calls shown inline with expandable details
• Agent Status Panel shows all agents with current task/status
• User can send messages into any task conversation
• User can create tasks manually
• Sub-tasks nested under parent tasks in sidebar
• All agent activity observable — no hidden state