architecture.md

Architecture

OpenBot combines a React app, a Hono API server, PostgreSQL, CopilotKit Intelligence, AG-UI Bot endpoints, and governed browser computers.

Regenerate it with bun run diagram after changing anything it shows.

Services and ports

Component	Port	Responsibility
app	3010	React/Vite interface for channels, Bot chat, live screen, settings, and admin pages.
server	3001	API, CopilotKit runtime, auth, roles, tenant package, coworkers, channels, policy, audit, credentials, plugins, components, and connectors.
agent-computer	4100	Chromium, /workspace, browser profile, screenshots, snapshots, and file tools.
agent-bot	4200	Proof-of-concept AG-UI Bot.
agent-langgraph	4201	LangGraph AG-UI Bot.
supervisor	4500 host / 4300 container	Creates, stops, resets, and lists per-Bot computer containers.
PostgreSQL with pgvector	5432	Product data, audit rows, credentials, policy, grants, channels, and components.
CopilotKit Intelligence	external	Durable threads, memory, and realtime gateway.

scripts/start.sh starts PostgreSQL, agent-computer, agent-bot, agent-langgraph, and the supervisor through Docker Compose, then starts server and app on the host.

The compose file also defines optional SPIRE services. start.sh does not start them.

Runtime flow

1. The app opens a channel or direct Bot session.
2. The server resolves the signed-in actor and selected coworker.
3. CopilotKit runtime sends the turn to the configured AG-UI endpoint.
4. The surface registers available frontend tools: browser tools, MCP tools, and components granted to that Bot.
5. Acting browser/file/MCP calls return to the server for authorization and audit.
6. The server streams results back to the app and Intelligence thread.

Browser action governance

The computer itself does not decide policy. The server gateway is the action boundary:

1. resolve the target from the server-held snapshot or request subject;
2. evaluate the current action policy;
3. write an audit row for the decision;
4. call the computer only when the decision forwards;
5. write a second audit row if a forwarded action fails.

Policy rules can inspect:

• tool.name
• intent
• bot.id
• actor.id
• page.url, page.host
• element.ref, element.role, element.name, element.type
• key
• file.path, file.name, file.extension
• mcp.server, mcp.tool, mcp.effect

Rules use CEL expressions plus case-insensitive contains() and matches(). Deny rules are evaluated before allow rules. The policy engine fails closed: a missing or empty policy permits nothing, a broken deny rule denies, and a broken allow rule does not permit. OpenBot's shipped startup default is explicit: deny: [] and allow: ["true"], unless AGENT_COMPUTER_POLICY or a saved administrator policy replaces it. A malformed configured policy stops server startup.

Computers

agent-computer requires COMPUTER_TOKEN and permits only /health without it. Docker Compose binds it to 127.0.0.1:4100.

With COMPUTER_SUPERVISOR_URL, each Bot gets its own computer container, workspace volume, and browser profile. Without it, all Bots share AGENT_COMPUTER_URL.

A command on the computer inherits PATH, locale and terminal names, and the proxy variables, not the rest of the process environment. Userinfo is stripped from a proxy URL. COMPUTER_SHELL_ENV names anything else a deployment wants passed.

The supervisor exposes only ensure, stop, reset, and list operations. It holds the Docker socket, so do not expose it outside the deployment network. Set COMPUTER_RUNTIME=runsc to run computers under gVisor on hosts that support it.

Human control and secrets

Handovers are audited as control events:

• computer.help_requested
• computer.control_taken
• computer.control_released

While a person controls the browser, Bot actions are refused rather than queued.

Secret entry is separate from chat content. The audit trail records that a secret was requested or supplied and the character count, not the secret value.

Watching a Bot work

Two surfaces beside the conversation. The screen is the live browser, proxied over a websocket and gated on the same question as every other route about that Bot. The Activity tab is what the Bot did away from the browser: every command with its output and exit code, every file read, write and listing, newest first.

Activity is held in the browser for the open conversation and is gone on reload. It is a window rather than a record; the record is the audit trail, which is server-side, survives restarts, and is what an investigation reads. A saved file contributes its path and size and never its contents, matching the write route, which declines to echo them because a Bot may be saving something it was told in confidence.

Coworkers and channels

A coworker is a durable Bot profile:

• agents stores runtime identity and endpoint/key reference.
• agent_profiles stores name, title, role, owner, visibility, and deletion state.
• agent_preferences stores per-user roster state.

A channel is a conversation with one coworker and a CopilotKit Intelligence thread mapping. Starting a new channel creates a new thread.

Who may reach one is decided by membership: every channel route resolves the caller in channel_memberships and refuses without a row. channels.allowed_groups is declared in the tenant package and stored, and is not part of that decision — users.groups is never populated by any sign-in path, so a group-based rule has nothing to evaluate. Treat it as a declaration waiting on group membership from the identity provider, not as a control that is running.

See coworkers.md.

Components

Components are frontend tools a Bot can call instead of answering only in prose.

Sources:

• compiled React components in app/src/components/gallery/;
• sandboxed components authored and published from /admin/playground.

Governance:

• compiled components are published when first seen by the app catalogue sync;
• sandboxed components are saved as drafts and become usable only after publish;
• every call asks the server whether the component exists, is published, and is not withheld from the Bot;
• component data functions require a separate per-component grant.

The shipped component data functions read the audit trail: botActivity and recentRefusals.

MCP and skills

MCP servers and skills share the plugin grant table, but they have different ownership rules.

• MCP tools are admin-governed because they can reach external systems with stored credentials.
• Skills are reusable instructions. A person can create personal skills and attach them only to Bots they own. Administrators create deployment skills.

The curated MCP catalogue contains Google Drive. Custom MCP servers must pass URL checks; unknown tools and custom-server tools are treated as writes unless positively classified as reads.

A catalogue entry says whose credential a Bot reaches it with, which is a different question from whether it is reachable at all. A deployment-wide token answers the same for everybody; Google Drive is user-oauth, so a Bot reads it as the person asking and sees only what that person can see. An administrator enabling the connector and a person connecting their own account are two decisions, and neither can be made for the other. See Google Drive.

Every MCP call checks the grant first, then evaluates the same action policy engine with MCP context, then audits the result.

Tenant package and knowledge

TENANT_PACKAGE_DIR points at the tenant package. The default is ../examples/fintech.

Required package files:

• brand.yaml
• agents.yaml
• channels.yaml
• model.yaml
• knowledge.yaml

The server validates the package at startup. Channel agent IDs must match declared agents. Knowledge sources currently support Google Drive and Microsoft OneDrive declarations.

Connector credentials are stored through the credential vault and referenced by id, not stored inline in YAML.

Security boundaries

• Server routes enforce auth and roles; admin pages are backed by server-side administrator checks.
• Sign-in is Google, Microsoft or Okta from the environment, plus SAML and OpenID Connect providers registered at runtime and routed by email domain. One resolver answers both questions a run asks about a person, whose threads these are and which Bots they may run, so the two can never disagree.
• INITIAL_ADMIN_EMAILS is a floor: an address it names is made an administrator at every sign-in and cannot be demoted from the People screen. Everybody else's role is decided there, and every change writes an audit row.
• Registering, changing or removing an identity provider is administrator-only. Better Auth's SSO plugin guards those routes with a session alone, which would let any signed-in person register a provider for a domain.
• A registered identity provider belongs to the deployment, not to whoever registered it. Better Auth scopes its own listing and removal to the registering user and cascades the row from that user, so two administrators saw two different deployments and deleting the one who set sign-in up would have deleted the company's sign-in. Reads and removals go through OpenBot's own administrator-only routes against the whole table.
• A provider's client secret and SAML signing material are encrypted at rest with KEY_ENCRYPTION_KEY, through a wrapper on the Better Auth storage adapter, since the plugin stores them as plaintext JSON. OAuth access and refresh tokens use Better Auth's own encryption, keyed on BETTER_AUTH_SECRET.
• Signing in, being refused, and being granted the administrator role by configuration each write an audit row. Without them nothing recorded that somebody who could edit INITIAL_ADMIN_EMAILS had promoted themselves, and revoking a person deleted the sessions that were the only evidence they had been here.
• Removing somebody deletes their sessions and denies their address, because deleting the user row alone is not removal: the next sign-in through the provider recreates it.
• With no identity provider configured, the deployment refuses to start unless OPENBOT_SINGLE_USER=true says every request may be one fixed administrator. That flag is the only thing that permits it; NODE_ENV does not.
• KEY_ENCRYPTION_KEY must be a base64-encoded 32-byte value. The example key is refused with NODE_ENV=production.
• Credential plaintext is encrypted at rest, never returned by APIs, and redacted from audit events.
• Browser navigation allows http and https; cloud metadata addresses are refused under every configuration.
• AGENT_COMPUTER_ALLOW_PRIVATE_HOSTS=true is for local development only.
• Computer tokens and supervisor tokens must be long random values outside local development.