Realtime voice agent using Vercel AI Gateway with MOSS as the knowledge base. Speak a question — the agent searches your MOSS index and answers out loud.
Browser (useRealtime) ──WebSocket── Vercel AI Gateway ── gpt-realtime-2
│ │
│ tool call: search_knowledge_base │
└─── POST /api/token ───────────────►│
MOSS index (local)
POST /api/token(empty body) — mints a short-lived WebSocket token via the gatewayPOST /api/token({ query }) — executes MOSS search; uses local in-memory index loaded at startup
Security note:
/api/tokenis unauthenticated only whenALLOW_UNAUTHENTICATED_DEMO=trueis set (demo). Before deploying publicly, add a session/cookie check so arbitrary callers cannot mint Gateway tokens or query your index.
Requires Node.js ≥ 22 (ai@7 and @ai-sdk/gateway@4 require it).
npm installcp .env.example .env| Variable | Where to get it |
|---|---|
MOSS_PROJECT_ID |
moss.dev dashboard |
MOSS_PROJECT_KEY |
moss.dev dashboard |
MOSS_INDEX_NAME |
Name of the index to search |
AI_GATEWAY_API_KEY |
Vercel AI Gateway → API Keys |
npm run devOpen http://localhost:3000 and tap the orb to start talking.