Skip to content

page.mdx

Latest commit

 

History

History
161 lines (120 loc) · 4.96 KB

page.mdx

File metadata and controls

161 lines (120 loc) · 4.96 KB

Observability Dashboard

Monitor agent-browser sessions in real time with a local web dashboard showing a live browser viewport and command activity feed.

Usage

The dashboard is bundled into the binary and requires no separate install. Start the server and open any session:

agent-browser dashboard start
agent-browser open example.com

Then open http://localhost:4848 in your browser to see the live dashboard.

All sessions automatically stream to the dashboard. No extra flags are needed.

Custom stream port

By default each session binds its WebSocket stream server to an OS-assigned port. To use a specific port, set the AGENT_BROWSER_STREAM_PORT environment variable:

AGENT_BROWSER_STREAM_PORT=9223 agent-browser open example.com

You can also use the runtime commands to control streaming on a running session:

agent-browser stream enable --port 9223
agent-browser stream status
agent-browser stream disable

Dashboard features

The dashboard is a single-page web app with three areas:

Area Description
Live viewport Real-time JPEG frames from the browser, rendered to a canvas element
Activity feed Chronological stream of commands, results, and console messages with expandable details
Session creation Create new sessions from the dashboard with local engines (Chrome, Lightpanda) or cloud providers (AgentCore, Browserbase, Browserless, Browser Use, Kernel)
Status bar Connection status, viewport dimensions, and WebSocket endpoint

WebSocket protocol

The dashboard connects to the same WebSocket endpoint used by Streaming, with additional message types for observability:

Command events

Sent when a command begins executing:

{
  "type": "command",
  "action": "click",
  "id": "r123",
  "params": { "selector": "@e5" },
  "timestamp": 1711367000000
}

Result events

Sent when a command finishes:

{
  "type": "result",
  "id": "r123",
  "action": "click",
  "success": true,
  "data": {},
  "duration_ms": 45,
  "timestamp": 1711367000045
}

Console events

Sent when the browser logs to the console:

{
  "type": "console",
  "level": "log",
  "text": "Page loaded",
  "args": [{"type": "string", "value": "Page loaded"}],
  "timestamp": 1711367000100
}

The args array contains the raw CDP Runtime.consoleAPICalled arguments for programmatic access. Object arguments include preview data (e.g. {userId: "abc", count: 42} instead of "Object").

These are in addition to the existing frame, status, and error message types documented on the Streaming page.

Architecture

The dashboard is a Next.js static export (output: 'export') that produces plain HTML, CSS, and JS. It lives at packages/dashboard/ in the monorepo and is built with:

pnpm build:dashboard

The dashboard is embedded into the CLI binary at compile time using rust-embed. Plain HTTP requests serve the embedded dashboard assets, while WebSocket upgrade requests are handled as before.

AI Chat

The dashboard includes an optional AI chat panel powered by the Vercel AI Gateway. When enabled, a Chat tab appears in the right pane alongside Activity, Console, Network, Storage, and Extensions.

Setup

The Chat tab is always visible. Set the API key to enable responses:

export AI_GATEWAY_API_KEY=gw_your_key_here
agent-browser dashboard start

Optionally override the gateway URL or model:

export AI_GATEWAY_URL=https://ai-gateway.vercel.sh   # this is the default
export AI_GATEWAY_MODEL=openai/gpt-4o-mini           # default: anthropic/claude-sonnet-4.6

How it works

The Rust server proxies chat requests from the dashboard to the Vercel AI Gateway and streams responses back using the Vercel AI SDK's UI Message Stream protocol. The dashboard frontend uses useChat from @ai-sdk/react with DefaultChatTransport.

VariableDescriptionDefault
AI_GATEWAY_URLVercel AI Gateway base URL.https://ai-gateway.vercel.sh
AI_GATEWAY_API_KEYAPI key for the AI Gateway. Required to enable AI chat responses.(none)
AI_GATEWAY_MODELDefault AI model for chat requests.anthropic/claude-sonnet-4.6
EXA_API_KEYExa API key. When set, the AI can search the web via Exa instead of navigating to a search engine.(none)