Protocol Observatory

Protocol Observatory is an interactive playground designed to dissect and visualize the end-to-end lifecycle of LLM chat applications. It serves as a "microscope" for understanding streaming protocols, agentic workflows, rate limiting, and observability.

This project evolved from a simple visualization tool into a production-ready reference architecture, demonstrating how to build robust AI applications using modern web standards.

🛠 Tech Stack

Built on the T3 Stack, leveraging modern web standards for performance, type safety, and scalability.

• Framework: Next.js 15 (App Router)
• Language: TypeScript
• Styling: Tailwind CSS
• Database: PostgreSQL (via Prisma ORM)
• Auth: NextAuth.js (v5 Beta)
• API: Server-Sent Events (SSE) & tRPC
• Testing: Vitest

✨ Features

🔬 The Microscope

A specialized UI component that visualizes the hidden details of LLM interactions:

• Streaming Mechanics: Real-time visualization of Time-to-First-Byte (TTFB) and token generation rates.
• Agent Workflows: Support for multi-step "Thinking" processes vs. direct "Non-Thinking" responses.
• Protocol Analysis: Inspect how tool calls and orchestration affect the user experience.

� Finance Agent (Live)

• Single powerful entry point: Live mode is refocused to a personal finance + market analysis agent (internally tool-driven, externally one agent).
• Market + news tools: Supports market quote/history and web/news search tools, requiring traceable sources and no fabricated links/data.
• Personal data: Minimal profile and knowledge-card storage APIs (/api/finance/profile, /api/finance/cards) with user isolation and deletion.
• Teaching preserved: chat/agent/ide/cli remain as teaching Mock modes; real calls are enabled only in Finance mode.
  • Example: after sign-in, PUT /api/finance/profile with {"data":{"risk":"moderate","horizon_years":5,"goals":["retirement","emergency_fund"]}}
  • Example: POST /api/finance/cards with {"title":"Duration","content":"...","tags":["bond","risk"],"sourceUrls":["https://..."]}

��️ Backend & Infrastructure

• Hybrid Streaming Engine:
  • Mock Mode: Zero-latency simulation for UI testing (default).
  • Live Mode: Real-time OpenAI-compatible API proxy (requires Authentication).
• Robust Rate Limiting: Token bucket algorithm implementing strict quotas (default: 60 requests/hour/user) to prevent abuse.
• Observability:
  • Metrics: Real-time request counters and latency histograms exposed at /api/metrics.
  • Tracing: Detailed request-level tracing for debugging complex agent flows at /api/debug/traces.

📐 Architecture

The application follows a clean, unidirectional data flow:

1. Client: The Microscope component initiates a persistent SSE (Server-Sent Events) connection to /api/chat/stream.
2. Gateway: The Next.js API route authenticates the request via NextAuth and checks rate limits against the PostgreSQL database.
3. Engine:
   • If Mock: Generates synthetic tokens based on predefined scenarios.
   • If Live: Proxies the request to an OpenAI-compatible provider (default base URL points to DeepSeek), handling stream transformation and backpressure.
4. Observability: Side-effects record metrics and traces to an in-memory store and expose them via /api/metrics and /api/debug/traces.

🚀 Getting Started

Prerequisites

• Node.js 18+
• Docker (optional, for local Database)

Installation

1. Clone and Install

   git clone https://github.com/FrankieLiu04/how-agent-work.git
   cd how-agent-work
   npm install

2. Initialize Database Start a local PostgreSQL instance (or provide your own DATABASE_URL):

   ./start-database.sh

3. Configure Environment

   cp .env.example .env

   Edit .env and fill in the required values (see Configuration).

4. Run Migrations

   npm run db:migrate

5. Start Development Server

   npm run dev

   Visit http://localhost:3000 to explore.

⚙️ Configuration

See .env.example for the full list of variables.

Variable	Description	Required
DATABASE_URL	PostgreSQL connection string	Yes
AUTH_SECRET	NextAuth secret key (generate with openssl rand -base64 32)	Yes
AUTH_GITHUB_ID	GitHub OAuth Client ID	Yes
AUTH_GITHUB_SECRET	GitHub OAuth Client Secret	Yes
AUTH_URL	NextAuth base URL (recommended for Vercel/custom domains)	No
AUTH_TRUST_HOST	Trust proxy headers (recommended on Vercel)	No
OPENAI_API_KEY	API key for live mode (OpenAI-compatible)	No
OPENAI_BASE_URL	Custom OpenAI-compatible base URL	No
TAVILY_API_KEY	Enables web search tool (agent)	No

📚 Documentation

• Project Overview: PROJECT.md
• Ops / Pitfalls Notes: AGENT.md
• Environment Variables: .env.example

🛠 Development Commands

npm run test        # Run unit tests
npm run typecheck   # Run TypeScript type checking
npm run build       # Build for production