Skip to content

architecture.md

Latest commit

 

History

History
210 lines (160 loc) · 8.84 KB

architecture.md

File metadata and controls

210 lines (160 loc) · 8.84 KB
Raw

Architecture

W Agent is a layered, self-hosted pipeline: a WhatsApp bridge normalizes events into a shared message model; queues make ingest and enrichment durable; storage owns history and outbox state; the agent decides when to act and how to draft; MCP exposes the same capabilities to external clients. Nothing above the bridge depends on Baileys or Meta-specific types.

Design goals

  1. Local control of history — chat data lands in your Postgres, not a vendor inbox.
  2. Human authority over sends — outbound traffic is staged (outbox) unless a chat is explicitly marked auto-send.
  3. Swappable WhatsApp transport — personal multi-device and official Cloud API share one interface.
  4. Fail closed on abuse — injection classification and outbound chat locks prefer silence over clever replies.
  5. Operable in production — watchdog, health endpoints, atomic auth writes, rate limits.

System context

flowchart LR
  Phone[Phone / WhatsApp]
  Meta[Meta Graph + webhooks]
  App[W Agent process]
  PG[(Postgres)]
  Redis[(Redis)]
  LLM[OpenAI APIs]
  Humans[Owner + MCP clients]

  Phone <-->|Baileys| App
  Meta <-->|Cloud API| App
  App --> PG
  App --> Redis
  App --> LLM
  Humans --> App
Loading

Layers

1. Bridge (src/bridge)

Responsibility: session lifecycle, inbound/outbound I/O, normalization to ProviderMessage.

Piece Role
WhatsAppProvider Interface: connect, disconnect, restart, sendText, sendMedia?, sendTyping?, getChats, getHistory, events
BaileysProvider Unofficial multi-device Web session; QR pairing; history sync hooks
CloudApiProvider Official Business Cloud API; webhook verify + POST; Graph text/templates
createProvider() Factory from WHATSAPP_PROVIDER
Filters / normalize Skip status/newsletters; map Baileys or Cloud payloads → ProviderMessage
Atomic auth Baileys multi-file state written via temp + rename + .bak

The bridge emits:

  • qr — pairing material (Baileys)
  • connected / disconnected — session health
  • message — normalized inbound (and some outbound) events

Higher layers subscribe with onMessage / attachIngestPipeline. They never import @whiskeysockets/baileys or Graph payload shapes.

2. Queues (src/queue)

Responsibility: durable, retryable work off the hot path of the socket/webhook.

Queue Worker Job
ingest ingest-worker Persist contact/chat/message; optional media download + enrichment; enqueue embed/summary
embed embed-worker Batch OpenAI embeddings → embeddings table
summary summary-worker Rolling per-chat memory in agent_memory

Redis (BullMQ) is the broker. The outbox sender is a lightweight poller in the main process (not a BullMQ worker): it drains approved rows through the send rate limiter and provider.sendText.

Inbound path:

ProviderMessage
  → toIngestJobData (filters)
  → enqueue ingest
  → persistInboundMessage
  → enqueue embed (+ maybe summary)

3. Storage (src/storage)

Responsibility: schema-backed persistence and query APIs used by agent, MCP, and dashboard.

Domain Examples
Identity contacts, chats
History messages (unique wa_message_id), media paths
Retrieval embeddings (pgvector), hybrid searchMessages, getChatContext
Memory agent_memory summaries
Outbox outbox statuses: pending → approved → sent / rejected / failed
Policy settings, chat_engagement, chat prefs (auto-send)

Postgres is the system of record. Redis is ephemeral (queues, bridge status, ops alerts), not the chat archive.

4. Agent + safety (src/agent, src/safety)

Responsibility: decide whether to engage, run a tool-using model loop, and stage replies safely.

sequenceDiagram
  participant In as Inbound message
  participant E as Engagement
  participant G as Guard
  participant A as Agent loop
  participant O as Outbox
  participant Owner as Owner WhatsApp

  In->>E: evaluate allowlist / mention / opt-out
  alt silent or escalate
    E-->>In: no agent turn
  else reply
    E->>G: classify injection
    alt forced silent
      G->>Owner: security alert
    else allowed
      G->>A: delimited untrusted content
      A->>A: tools search / context / summary
      A->>O: draft_reply
      O->>Owner: "Draft for …" notify
    end
  end
  Owner->>O: approve (dashboard / API)
  O->>In: sendText via provider
Loading
Module Role
engage.ts Allowlists, group mentions, /agent owner commands, debounce
guard.ts Untrusted delimiters, heuristics (+ optional LLM), outbound chat lock, secret redaction
core.ts Model-agnostic tool loop (ModelProvider)
tools.ts Search, context, summary, contacts, draft/send (send gated)
outbound.ts Insert outbox + owner self-chat notification
ratelimit.ts Quotas, quiet hours, typing delay, cooldown, daily cap

Owner /agent commands short-circuit engagement and run a trusted command path.

5. MCP (src/mcp)

Responsibility: expose read and draft capabilities to external MCP clients without a second product surface.

Surface Contents
Tools whatsapp_list_chats, whatsapp_get_history, whatsapp_search, whatsapp_get_summary, whatsapp_send_message (draft), whatsapp_contacts
Resources whatsapp://chats, whatsapp://chat/{jid}/history, whatsapp://chat/{jid}/summary
Prompts summarize_chat, draft_reply_in_my_tone, weekly_digest
Transports stdio (pnpm mcp), HTTP+SSE (MCP_PORT + MCP_AUTH_TOKEN)

MCP whatsapp_send_message uses the same draftReplyAndNotify path as the in-app agent. It does not bypass approval.

Client setup: mcp-clients.md.

6. Dashboard (dashboard/)

Next.js UI over the same Postgres/Redis data: pairing QR (from Redis bridge status), chats, prefs, approvals, health. Authenticated with DASHBOARD_TOKEN. It is a control plane, not a second message store.

7. Operations (src/ops)

Piece Role
Watchdog Periodic bridge / Redis / Postgres / queue checks; bridge restart with backoff; owner alert after N failures
/healthz Aggregated status for external uptime monitors
Alerts Structured log + Redis list + WhatsApp when the bridge is up

Provider-adapter rationale

WhatsApp access is the most volatile dependency in the system: unofficial sessions can be banned; the official Cloud API cannot sync personal history and requires business verification. Binding the agent, outbox, and MCP to a single SDK would force a rewrite when requirements change.

The WhatsAppProvider interface exists so that:

  1. Ingest and agent stay stable — they consume ProviderMessage and call sendText, nothing else.
  2. Risk can be chosen per deployment — Baileys for personal companion use (accept ban risk); meta for compliant business messaging.
  3. Parity can grow incrementally — Cloud API currently targets inbound text + outbound text/templates; Baileys carries history sync and richer media. Missing methods fail explicitly rather than leaking Baileys types upward.
  4. Tests prove the seam — mocked providers and the Cloud API suite exercise the same ingest → draft → send lifecycle.

Factory entrypoint: createProvider() in src/bridge/provider.ts. Selection: WHATSAPP_PROVIDER. Detailed trade-offs: providers.md.

Trust boundaries

Boundary Rule
Inbound WhatsApp body Untrusted data; delimited; classified before tool use
Owner /agent and dashboard actions Trusted control plane
MCP tools Same storage + draft APIs; remote HTTP requires bearer token
Model tool results Secret redaction before returning to the model
Auto-send Rate-limited; quiet hours; never owner-notify path

Threat model detail: security.md.

Repository map

src/
  bridge/     WhatsAppProvider implementations
  queue/      BullMQ producers/workers, outbox sender, ingest pipeline
  storage/    Postgres access, search, embeddings, outbox, settings
  agent/      Engagement, tools, model loop, outbound drafts
  safety/     Injection guard, send rate limiter
  mcp/        MCP server (stdio + HTTP/SSE)
  ops/        Watchdog, healthz, alerts
  index.ts    Process wiring
dashboard/    Next.js control UI
migrations/   SQL migrations (node-pg-migrate)
tests/        Unit + testcontainers integration
docs/         Architecture, security, MCP, providers

Related reading