Skip to content

Latest commit

 

History

History
1391 lines (1017 loc) · 83.3 KB

CONFIGURATION.md

File metadata and controls

1391 lines (1017 loc) · 83.3 KB

Configuration

Plexus does not support file-based application configuration. It does not load providers, models, keys, quotas, or settings from YAML, JSON, TOML, or other configuration files. Code examples in this document show request payload shapes only; apply them through the Admin UI or Management API.

Plexus stores all configuration in the database and manages it via the Admin UI (recommended) or Management API.

Environment variables control server-level settings. Everything else (providers, models, keys, quotas, and settings) is stored in the database.


Environment Variables

Variable Description Required
ADMIN_KEY Password for admin dashboard and management API. Server refuses to start if unset. Yes
DATABASE_URL Connection string. Supports sqlite:// and postgres:// URIs. No
ENCRYPTION_KEY 32-byte key for encrypting sensitive data at rest. Generated via: openssl rand -hex 32 No
DATA_DIR Directory for SQLite database. No
LOG_LEVEL Verbosity: error, warn, info, debug, silly No
PLEXUS_USAGE_RETENTION_DAYS Retention for request usage, debug, error, MCP, and quota meter-snapshot logs in days (default 365). Older rows are pruned daily. No
PORT HTTP server port (defaults to 4000; auto-derived from git worktree name when running bun run dev). No
HOST Address to bind to. No
FRPC_SERVER_ADDR Development-only LAN address of the frps server. No
FRPC_AUTH_TOKEN Development-only token shared with frps. No
FRPC_SERVER_PORT Development-only frps control port (defaults to 7000). No
FRPC_SUBDOMAIN_HOST Optional development-only host suffix used to print the full HTTPS tunnel URL. No

Quick Start

# SQLite (database auto-created in ./data/)
ADMIN_KEY="my-secret" bun run dev

# PostgreSQL
ADMIN_KEY="my-secret" DATABASE_URL="postgres://user:pass@localhost:5432/plexus" bun run dev

# Docker
docker run -e ADMIN_KEY="my-secret" -v ./data:/app/data -p 4000:4000 plexus:latest

Configuration via Admin UI

The Admin UI is accessible at the port printed by bun run dev after starting. It provides forms for all configuration options with real-time validation.

  • Providers: Add/edit upstream AI providers (API keys, base URLs, model lists)
  • Models: Create model aliases with routing logic and pricing
  • Keys: Manage client API keys with optional quota assignment
  • Quotas: Define usage limits (tokens, requests, or spending) per time window
  • Custom Quota Checkers: Write JavaScript integrations for provider quota APIs
  • MCP Servers: Configure MCP proxy endpoints
  • OAuth: Login to OAuth-backed providers (Anthropic, GitHub Copilot, Codex, etc.)
  • Settings: Vision fallthrough, global defaults, cooldown configuration

Management API

For programmatic configuration, use the Management API (/v0/management/*). All endpoints require the x-admin-key header.

Endpoint Description
GET /v0/management/providers List all providers
PUT /v0/management/providers/{slug} Create/update provider
DELETE /v0/management/providers/{slug} Remove provider
GET /v0/management/aliases List all model aliases
PUT /v0/management/aliases/{slug} Create/update alias
DELETE /v0/management/aliases/{slug} Remove alias
GET /v0/management/keys List all API keys
PUT /v0/management/keys/{name} Create/update key
DELETE /v0/management/keys/{name} Remove key
GET /v0/management/debug Read in-memory debug capture state
PATCH /v0/management/debug Update in-memory global/key/alias/provider debug capture targets
GET /v0/management/user-quotas List quota definitions
PUT /v0/management/user-quotas/{name} Create/update quota
DELETE /v0/management/user-quotas/{name} Remove quota
GET /v0/management/custom-checkers List custom provider quota checkers
PUT /v0/management/custom-checkers/{id} Create/update a custom quota checker
POST /v0/management/custom-checkers/{id}/test Test custom checker code without persisting a snapshot
DELETE /v0/management/custom-checkers/{id} Remove a custom quota checker
GET /v0/management/config/export Export full config as JSON
PUT /v0/management/config Import config (replace all)

See the API Reference for complete endpoint documentation.

For the complete custom checker workflow, context API, authentication/header configuration, and examples, see Custom Quota Checkers.

Debug Trace Capture

Debug trace targeting is runtime-only. GET /v0/management/debug returns the current in-memory state, and PATCH /v0/management/debug can update:

{
  "enabled": false,
  "keys": ["mobile-app"],
  "aliases": ["gpt-4o-mini"],
  "providers": ["openai"]
}

Capture is inclusive: a request is recorded when any enabled dimension matches the request key, canonical model alias, selected provider, or global flag. Setting providers does not filter out global/key/alias capture. keys, aliases, and providers can be set to null or [] to clear that target list.


Providers

A provider represents an upstream AI service that Plexus routes requests to. Each provider has authentication credentials, a base URL, and a list of available models.

Provider Settings

Setting Description Required
Slug Unique identifier (e.g., openai_direct, anthropic-prod) Yes
Display Name Friendly name for logs and UI No
API Base URL Provider's endpoint. Common values: Yes
https://api.openai.com/v1
https://api.anthropic.com/v1
https://generativelanguage.googleapis.com/v1beta
https://openrouter.ai/api/v1
oauth:// (for OAuth-backed providers)
API Key Authentication token Yes
Enabled Whether this provider is active for routing No (default: true)
Headers Custom HTTP headers sent with every request No
Extra Body Additional fields merged into every request No
Upstream Timeout Per-provider request timeout override in milliseconds. If unset, the global timeout is used. No
Disable Cooldown Exclude from automatic cooldown on errors No
Allow 100% Utilization Allow quota usage to reach 100% instead of cooling down at 99% No (default: false)
Stall Detection Overrides Optional per-provider overrides for TTFB/throughput stall detection. Empty = inherit global setting for that field. No
pi-ai Provider Builtin pi-ai provider ID used for registry model lookup (for example, anthropic, openai, google) No
pi-ai Quirks Inline compatibility traits by API and model, an alternative to pi-ai Provider. No
Auto Compat Opt-in reasoning and generation mapping from either a pi-ai model link or inline quirks. With neither, no registry-driven quirk handling occurs. No (default: false)
Adapters Request/response rewrite hooks applied to every model under this provider (see Provider Adapters) No
Native Responses Extensions Responses API extensions this provider accepts verbatim (see Responses API Extensions) No

Multi-Protocol Providers

Some providers support multiple API formats (OpenAI chat, Anthropic messages, embeddings). Configure them with a map of protocol → URL:

Protocol Use Case
chat OpenAI-compatible chat completions
messages Anthropic Claude Messages API
embeddings OpenAI-compatible embeddings (Gemini providers auto-transformed)
openai-images OpenAI-compatible image generation and edits (/images/generations, /images/edits)
openrouter-images Dedicated OpenRouter image endpoint (/images)
codex-images ChatGPT-subscription Codex Images backend; only valid on an openai-codex OAuth provider
transcriptions Speech-to-text (Whisper)
speech Text-to-speech

When combined with priority: api_match on a model alias, Plexus prefers providers that natively support the incoming API format.

An incoming /v1/images request can also be served by a chat or gemini target when the provider model declares it through access_via — Plexus picks the image transformer from the resolved target API type, not from the incoming request shape. The api_base_url map form must hold real URLs; an OAuth provider declares oauth:// through the string form instead.

OAuth Providers

Plexus supports OAuth-backed providers via the pi-ai library. Every OAuth-capable provider pi-ai ships is supported — new flows pi-ai adds become available without a Plexus update. These require authentication through the Admin UI.

Supported OAuth providers (as of the current pi-ai dependency):

  • Anthropic Claude
  • GitHub Copilot
  • OpenAI Codex
  • OpenAI o1-pro
  • xAI (Grok / X Premium+ subscription)
  • Kimi Code (Moonshot subscription)
  • OpenRouter

radius is the one pi-ai OAuth flow Plexus excludes — it's a configurable gateway factory rather than a fixed identity provider, so it doesn't fit this list. The current set is always available from the Admin UI's OAuth provider dropdown, or via GET /v0/management/oauth/providers.

Configuration:

  • Set API Base URL to oauth://
  • Set API Key to oauth
  • Set OAuth Provider if the provider key differs from pi-ai's expected ID

The OAuth account is the provider ID itself (one login per provider) — there is no separate account field. Existing logins created under older account names keep working.

Once configured, log in via the Admin UI to authorize Plexus. Tokens are stored encrypted (when ENCRYPTION_KEY is set) and auto-refreshed.

Codex Image Models

An openai-codex OAuth provider can also serve image generation through the ChatGPT Images backend. Mark the provider model type: image and give it access_via: ["codex-images"]. That is the only image protocol an OAuth route accepts; any other image target on an OAuth provider is rejected with a 400.

providers:
  codex:
    api_base_url: 'oauth://'
    api_key: 'oauth'
    oauth_provider: 'openai-codex'
    models:
      gpt-5.5: {}
      gpt-image-2:
        type: image
        access_via: ['codex-images']
        pricing: { source: per_request, amount: 0 }
        extraBody: { quality: low }
      gpt-image-2.5-flare:
        type: image
        access_via: ['codex-images']
models:
  codex-image:
    type: image
    targets: [{ provider: codex, model: gpt-image-2 }]

Image generation on a ChatGPT subscription is not billed per token, so give these models pricing with source: per_request and amount: 0. Usage records then carry a real cost of zero instead of a token-rate estimate.

extraBody is merged over the built image payload — provider first, then provider model, then alias, with later entries winning — so its values override any matching field the client sent, they do not merely fill in unset ones. The extraBody: { quality: low } above pins every request on this model to quality: low, including one that asked for quality: high. Use it to enforce a setting, and leave it out when clients should be able to choose.

The Codex target accepts size, n, quality, and background, and returns base64 image data. It rejects response_format: "url", output_format, output_compression, seed, style, user, and mask with a 400, and stream: true with a 501. Requests carrying reference images become Codex edits, with at most five references per request.

Reaching an Image Model from a Chat Client

An alias with type: image also answers chat-shaped requests. A request on /v1/chat/completions, /v1/responses, /v1/messages, the Gemini surface, or /v1/completions whose model resolves to an image model is turned into an image generation: the last user message becomes the prompt, that message's attached images become edit references, and n is 1. Size, quality, background, and output format stay unset, so provider/model/alias extraBody still decides them.

Responses-format clients receive native image_generation_call output items; chat, Anthropic Messages, Gemini, Ollama, and legacy Completions clients receive a markdown data URI. Routing, key access policy, quota, cooldown, failover, and usage recording are the same as for /v1/images/generations.

Set type: image on the alias when its targets are not uniformly image models: an alias that fans out across image and non-image targets with no alias-level type is left as an ordinary chat request, and Plexus logs a warning naming the alias. An explicit image_generation tool on an ordinary chat model is unaffected — detection never reads the request's tools.

Registry-Aware Compatibility

Plexus maps client reasoning and generation options using either a builtin pi-ai model link or explicit inline quirks. This preserves the v1 pass-through path and does not re-enable the removed inference-v2 path.

With a builtin link, set pi_ai_provider on the provider and pi_ai_model_id on each configured provider model. Enable auto_compat: true on the provider or model. If the pair is missing or unresolved, the registry rewrite does nothing (one exception below); unresolved configured pairs produce a startup warning.

For providers without pi-ai definitions, use pi_ai_quirks instead of pi_ai_provider. Keys are configured target APIs (chat, completions, messages, responses, gemini). Each entry specifies its pi-ai API dialect and only known traits: reasoning, thinkingLevelMap, maxTokens, and a bounded compat object. Optional models entries use exact upstream model IDs and override the common traits. Model-specific maps replace the common thinking-level map; individual compat flags merge. No pi_ai_model_id is needed. Unknown traits remain unknown and are not advertised or rewritten.

Some thinking endpoints (DeepSeek, Kimi K3, MiMo) reject a replayed history when any assistant message lacks reasoning_content. When the resolved pi-ai model declares requiresReasoningContentOnAssistantMessages — as the builtin DeepSeek models do — auto-compat fills reasoning_content: "" on assistant turns that recorded no reasoning. Recorded reasoning is preserved verbatim; no adapter option is needed. Uncatalogued chat models can declare the same trait inline (compat.requiresReasoningContentOnAssistantMessages: true, with reasoning: true). As a fallback for DeepSeek routes whose model ID is not in the catalog (for example deepseek-v4-flash), a deepseek.com base URL, the deepseek pi-ai provider link, or the deepseek provider slug enables the same fill whenever thinking is engaged. The built-in DeepSeek preset already enables auto-compat, so preset-created providers get this by default.

With neither source, requests use ordinary routing. Existing provider configs with auto_compat: true but no source remain valid and inert; new presets cannot enable auto-compat without a quirk source.

PUT /v0/management/providers/anthropic_oauth
{
  "api_base_url": "oauth://",
  "api_key": "oauth",
  "oauth_provider": "anthropic",
  "pi_ai_provider": "anthropic",
  "auto_compat": true,
  "models": {
    "claude-opus-4-6": {
      "type": "text",
      "pi_ai_model_id": "claude-opus-4-6"
    }
  }
}

You can also enable compatibility only for selected models:

{
  "pi_ai_provider": "anthropic",
  "models": {
    "claude-opus-4-6": {
      "type": "text",
      "pi_ai_model_id": "claude-opus-4-6",
      "auto_compat": true
    }
  }
}

For example, an OpenAI-compatible chat API that requires max_completion_tokens instead of max_tokens can declare just that quirk:

{
  "api_base_url": { "chat": "https://example.test/v1" },
  "api_key": "sk-example",
  "auto_compat": true,
  "pi_ai_quirks": {
    "chat": {
      "api": "openai-completions",
      "compat": { "maxTokensField": "max_completion_tokens" }
    }
  },
  "models": { "upstream/model-id": {} }
}

The preset catalog at provider-presets.json uses camelCase (piAiProvider or piAiQuirks) and publishes an editor JSON Schema. An inline profile is saved into the provider config, so later remote catalog edits do not change configured behavior. Model discovery and model-listing URLs are separate work.

reasoning_rewrite remains a manual escape hatch but overlaps with auto-compat. Avoid enabling both for the same model and field.

Responses API Extensions

Agent clients such as Codex CLI and Muse Code send OpenAI Responses API extensions that many Responses-compatible upstreams reject:

Extension Wire shape
namespace_tools tools[] entries of type namespace grouping sub-tools
namespaced_calls function_call history with a namespace field ({namespace, name})
dotted_calls function_call history named <namespace>.<tool> (Muse Code's muse.bash)
custom_tools tools[] entries of type custom taking raw string input (e.g. apply_patch)
custom_calls custom_tool_call / custom_tool_call_output history items
additional_tools additional_tools input items (Codex lite mode)
tool_search client-executed tool_search tools and history items

Plexus forwards a Responses request verbatim only when the target accepts every extension the request carries. Otherwise it flattens them: namespace tools become namespace__name function tools, custom tools become functions with a single string input, and tool calls are split back into the client's original shape on the response. Any client can use any Responses or Chat target this way.

For Codex OAuth, supported same-format Responses requests retain their original fields. Plexus wraps string input in a message, converts system input messages to developer messages, applies provider compatibility settings and adds fields required by the Codex backend. Unsupported extensions and cross-format requests use transformation.

The accepted set depends on the target:

  • responses:lite targets use the fixed lite contract: every extension except top-level namespace tools and dotted names. The upstream enforces it, so the provider setting doesn't change it.
  • Plain responses targets use the provider's Native Responses Extensions setting (responses_extensions) when set (an empty list flattens everything), otherwise the provider default, from its OAuth provider or the host of its Responses endpoint:
Provider Default
Codex OAuth everything except dotted_calls
Meta OAuth, or a Responses URL on api.meta.ai namespace_tools, namespaced_calls, dotted_calls
Responses URL on api.openai.com namespace_tools, namespaced_calls, custom_tools, custom_calls
Anything else custom_tools

The defaults track the endpoint, so a provider pointed at api.openai.com or api.meta.ai needs no configuration. Set the field only to override a default or to opt a compatible third-party endpoint in, for example:

{
  "api_base_url": { "responses": "https://gateway.example.com/v1" },
  "responses_extensions": ["namespace_tools", "namespaced_calls"]
}

Raw Provider Passthrough

Static API-key providers can expose an opt-in raw endpoint that bypasses model routing, failover, adapters, and request/response transformation:

PUT /v0/management/providers/openrouter
{
  "api_base_url": "https://openrouter.ai/api/v1",
  "api_key": "sk-or-...",
  "raw_passthrough": {
    "enabled": true,
    "base_url": "https://openrouter.ai/api",
    "auth": "bearer"
  }
}

POST /raw/openrouter/v1/responses?trace=true is sent to https://openrouter.ai/api/v1/responses?trace=true. The method, query string, request body, upstream status, response headers, and response body are relayed without application-level transformation. Plexus client credentials are removed and the request Host is selected from the configured upstream URL; other client headers are preserved. The provider's configured API key is inserted using bearer, x-api-key, or x-goog-api-key authentication. Responses preserve upstream headers and add x-request-id, x-plexus-request-id, and applicable quota metadata. For recognized Chat Completions, Messages, Responses, and Gemini paths, Plexus also feeds the unmodified response through its existing debug reconstruction and usage normalization pipeline to record the requested model, tokens, reasoning/cache usage, and provider-reported costs when present. This observation does not alter bytes sent in either direction.

Raw access is default-deny and requires allowRawPassthrough: true on the calling key. The key's existing provider allow/deny lists still apply, but model allow/deny lists do not: raw access is a privileged provider-wide capability. Request-count quotas can be enforced by provider; model-scoped and token/cost enforcement cannot be guaranteed for opaque traffic. OAuth providers are not supported by the raw endpoint.

Provider Adapters

Adapters rewrite request payloads outbound to a provider and raw response payloads inbound, fixing provider-specific field-name incompatibilities without modifying the core transformer pipeline.

Adapters can be set at provider level (applied to every model under the provider) or at model level (appended after provider-level adapters for a specific model). Both accept a single name or a list. Use { "name": "...", "enabled": false } at provider or model level to disable an inherited or automatic adapter; a later enabled entry restores it.

Adapter Description
reasoning_content Renames reasoning / thinking.content → reasoning_content on outbound assistant messages for providers that use Fireworks/DeepSeek field naming (e.g. Fireworks DeepSeek-R1). Fixes "Extra inputs are not permitted, field: messages[N].reasoning" errors.
suppress_developer_role Rewrites the developer role to system on outbound messages for providers that do not support the newer OpenAI developer role.
model_override Conditionally rewrites the provider model name based on request payload fields. Used for providers that expose reasoning variants as separate model names rather than respecting reasoning-related fields in the request body. See Model Override Adapter below.
reasoning_rewrite Manually rewrites reasoning/thinking request fields for providers with bespoke compatibility requirements. Prefer auto_compat for models linked to the pi-ai builtin registry; this adapter is now best treated as an escape hatch.
web_search_coercion Translates server-side web search tool entries to the format expected by the target provider. Clients can use any web search format; Plexus rewrites it transparently. See Web Search Coercion Adapter below.
suppress_unsupported_gpt5_options Removes generation options unsupported by GPT-5 models. Enabled automatically for the GPT-5 family; set enabled: false for a model to opt out.
normalize_anthropic_tool_ids Rewrites tool_use / tool_result ids that violate Anthropic's ^[a-zA-Z0-9_-]+$ charset (e.g. Moonshot's functions.WebSearch:3), which Anthropic rejects with a hard HTTP 400. Enabled automatically when the outbound wire format is Anthropic Messages and the provider looks Anthropic (base URL containing anthropic.com, Anthropic OAuth, or Claude masking). Add { "name": "normalize_anthropic_tool_ids", "options": {}, "enabled": true } to force it on for an Anthropic-compatible gateway hosted on another domain; set enabled: false to opt out.
drop_unsigned_thinking Removes thinking blocks that carry no usable signature from outbound Anthropic Messages bodies — unsigned reasoning reaches them through a cross-format replay (reasoning_content mapped to thinking) or a client forwarding a history it never signed, and Anthropic rejects every such turn with a hard HTTP 400. A turn left with no content is dropped and same-role neighbours are merged so the roles keep alternating. When a manual extended-thinking request (thinking: { "type": "enabled" }) is left in an open tool-use cycle whose assistant turn no longer opens with a thinking block after the strip, the request-level thinking config and its output_config.effort are removed as well: Anthropic demands a leading thinking block there ("a final assistant message must start with a thinking block"), which the strip has just made impossible. Completed turns followed by a fresh user turn keep their config. Enabled automatically under the same gate as normalize_anthropic_tool_ids; set { "name": "drop_unsigned_thinking", "options": {}, "enabled": false } to opt out (for a detected Anthropic route whose gateway wants unsigned thinking kept), or enabled: true to force it on elsewhere.

Example — provider-level:

PUT /v0/management/providers/fireworks
{
  "api_base_url": "https://api.fireworks.ai/inference/v1",
  "api_key": "fw_...",
  "adapter": "reasoning_content"
}

Example — model-level override:

{
  "models": {
    "accounts/fireworks/models/deepseek-r1": {
      "adapter": ["reasoning_content", "suppress_developer_role"]
    }
  }
}

Adapters are applied in order on outbound (preDispatch) and in reverse on inbound (postDispatch). They run whether or not the pass-through optimisation is active: a same-format request skips the transformer, but its body is still cloned, given the target model and the provider/model config fields, and run through the adapter chain before dispatch.

Model Override Adapter

The model_override adapter conditionally rewrites the provider model name based on the values or presence of fields in the request payload. This is useful for providers that expose reasoning variants as separate model names (e.g. model-name with reasoning, model-name-fast without) rather than respecting reasoning-related fields in the request body.

How it works:

When the resolved provider model matches a rule's model field AND any of the rule's conditions are satisfied (OR semantics), the model name is rewritten to rewriteTo. Conditions use dotted paths into the request payload.

Configuration:

The model_override adapter is configured at model level only (not provider level). It accepts a rules array in its options:

{
  "models": {
    "zai-org/GLM-5.1-FP8": {
      "adapter": [
        {
          "name": "model_override",
          "options": {
            "rules": [
              {
                "model": "zai-org/GLM-5.1-FP8",
                "rewriteTo": "glm-5.1-fast",
                "conditions": [
                  { "field": "enable_thinking", "value": false },
                  { "field": "reasoning.enabled", "value": false },
                  { "field": "reasoning.effort", "value": "none" },
                  { "field": "budget_tokens", "value": 0 }
                ]
              }
            ]
          }
        }
      ]
    }
  }
}

Rule fields:

Field Description
model The provider model name to match against (must match the resolved target model)
rewriteTo The model name to send to the provider instead
conditions Array of conditions; any match triggers the rewrite

Condition fields:

Field Description
field Dotted path into the request payload (e.g. reasoning.enabled, chat_template_kwargs.enable_thinking)
value Value to match (strict equality). If omitted, the condition matches when the field is present (any value)

Notes:

  • The adapter operates on the transformed provider payload, so fields must survive API transformation to be matchable. For chat-to-chat requests, all fields from the original request body are preserved (including non-standard fields like enable_thinking, thinking_budget, etc.).
  • Multiple rules are evaluated in order; only the first matching rule applies.
  • The rewrite is transparent to the client — billing and usage tracking still reference the original canonical model name.

Web Search Coercion Adapter

Different providers expose server-side web search under completely different tool type strings, making it impossible for a client to use a single request format across all providers. The web_search_coercion adapter solves this: configure the target format once on the provider, and Plexus rewrites every web search tool entry in the outgoing request automatically — regardless of which format the client sent.

Supported incoming formats (any of these will be recognised and coerced):

Provider Tool entry
Anthropic { "type": "web_search_20250305", "name": "web_search", "max_uses": N }
OpenAI { "type": "web_search" }
OpenRouter { "type": "openrouter:web_search" }
Google { "googleSearch": {} } / { "type": "googleSearch" }

Supported targets (the format Plexus rewrites to):

target Output tool entry Provider
anthropic { "type": "web_search_20250305", "name": "web_search" } Anthropic API
openai { "type": "web_search" } OpenAI Responses API
openrouter { "type": "openrouter:web_search" } OpenRouter
google { "googleSearch": {} } Google Gemini native API

The adapter operates transparently across all four incoming API surfaces (Chat Completions, Anthropic Messages, OpenAI Responses, and Gemini native) — a client using any of these APIs and any web search format will have its request correctly rewritten for the target provider.

Configuration:

Set the adapter on the provider that needs coercion. The only required option is target:

PATCH /v0/management/providers/my-openrouter-provider
{
  "adapter": [
    {
      "name": "web_search_coercion",
      "options": {
        "target": "openrouter"
      }
    }
  ]
}

For the Anthropic target, an optional max_uses limits how many web searches are allowed per request:

{
  "name": "web_search_coercion",
  "options": {
    "target": "anthropic",
    "max_uses": 5
  }
}

Options:

Option Type Required Description
target "anthropic" | "openai" | "openrouter" | "google" Yes The provider's expected web search tool format
max_uses integer No Maximum web searches per request. Anthropic target only; ignored for other targets.

Notes:

  • Non-web-search tools (regular function tools) in the same request are left untouched.
  • The adapter only fires when a web search tool is present; requests without web search tools have zero overhead.
  • For Google providers, web search coercion requires the native Gemini API endpoint (/v1beta/models/...). The OpenAI-compatible endpoint Google exposes does not support any web search tool format.
  • Pass-through optimisation is automatically disabled when any adapter is active.

Provider Quota Checkers

Quota checkers monitor upstream provider rate limits and prevent routing to exhausted providers.

Checker Type Description Options
synthetic Usage from Synthetic API apiKey (defaults to provider's key)
naga Naga AI balance
nanogpt NanoGPT usage
openai-codex Codex quota (OAuth) Reads token from database
claude-code Claude Code quota (OAuth) Reads token from database
zai ZAI balance
moonshot Moonshot balance
novita Novita balance
minimax Minimax balance Requires groupid, hertzSession

The claude-code checker reports available rolling usage windows and enabled monthly USD extra-usage limits. When rolling windows are present, the extra-usage meter is informational and does not trigger provider cooldown. When spend is the only meter, it can trigger cooldown.

Settings:

  • enabled: Enable/disable polling
  • intervalMinutes: Polling frequency (minimum 1)
  • maxUtilizationPercent: Treat provider as exhausted when any window reaches this % (default 99)

Quota data is available via the Management API — see API Reference: Quota Management.

Provider Timeout Overrides

Each provider can optionally set timeoutMs to override the global upstream timeout for requests routed to that provider.

  • Unset / omitted: inherit the global timeout
  • Set to a number: use that provider-specific timeout instead
  • Valid range: any positive integer millisecond value in provider config; the Admin UI exposes this as 1–3600 seconds

Use provider overrides when one backend is predictably slower or faster than the rest. For example, you might keep a 300s global timeout but set a 30s timeout on a fast inference endpoint so stuck requests fail over much sooner.

Provider Stall Detection Overrides

Each provider can optionally override any of the global stall detection settings with these fields:

  • stallTtfbMs — per-provider TTFB timeout override
  • stallTtfbBytes — per-provider TTFB byte threshold override
  • stallMinBps — per-provider minimum throughput override
  • stallWindowMs — per-provider sliding-window width override
  • stallGracePeriodMs — per-provider grace period override

Important inheritance rules:

  • If a field is omitted, the provider inherits the global setting for that field.
  • For the nullable threshold fields (stallTtfbMs, stallMinBps), setting the value to null disables that stall dimension for the provider.
  • For the non-null tuning fields (stallTtfbBytes, stallWindowMs, stallGracePeriodMs), null/empty in practice means “use the inherited global value”.

This lets you keep one global policy while tightening or relaxing stall protection for known outliers.


Model Aliases

A model alias is a virtual model name that clients use in requests. Each alias maps to one or more provider targets with routing logic.

Alias Settings

Setting Description Required
Slug Name clients send (e.g., fast-model) Yes
Type chat (default), embeddings, transcriptions, speech, image No
Additional Aliases Alternative names that also route here No
Priority Routing order: selector (default) or api_match No
Target Groups Ordered groups of targets, each with its own selector Yes
Metadata External catalog for model info No

Target Groups

Aliases contain one or more target groups. The dispatcher exhausts all healthy targets in group 1 before trying group 2, and so on. Each group has:

  • Name — Label for the group (e.g. "subscription", "payg", "backup")
  • Selector — Strategy for ordering targets within this group
  • Targets — List of provider/model pairs in this group

This allows you to organise targets by preference. For example:

Group Name Selector Purpose
1 subscription e2e_performance Use paid subscriptions first (sunk cost)
2 payg cost Fall back to cheapest pay-as-you-go option
3 backup in_order Last-resort backup target

If no groups are explicitly configured, all targets live in a single default group.

Selector Strategies

Strategy Behavior
random (default) Distributes requests randomly across healthy targets in the group
in_order Tries targets in order, skips unhealthy ones
cost Routes to cheapest provider (requires pricing)
performance Routes to highest post-TTFT throughput (output tokens / streaming time)
latency Routes to lowest time-to-first-token
usage Routes to provider with least recent usage (last 24 hours)
quota Favors tracked targets with the most quota headroom to consume before reset; utilization is scored continuously, while non-expiring balances rank after resettable quotas
e2e_performance Routes to highest end-to-end throughput (output tokens / total request time)

Inline Exploration (default)

Use performanceExplorationRate (default 0.05) to occasionally explore other targets and prevent locking onto one provider. Applies to performance, latency, and e2e_performance selectors. latencyExplorationRate and e2ePerformanceExplorationRate can be set separately for their respective selectors (each defaults to performanceExplorationRate if not specified). Unlike performance, the e2e_performance selector explores across all candidates including the current best, ensuring end-to-end metrics stay fresh for every provider.

Inline exploration occurs on the live request path: a small fraction of incoming requests are intentionally routed to non-optimal targets so their performance data stays fresh. The trade-off is that those specific requests may be slower than necessary.

Background Exploration

To keep performance data fresh without ever diverting live requests, enable backgroundExploration instead. When enabled, the inline exploration above is suppressed for the three perf-based selectors, and Plexus instead fires small representative probe requests in the background against stale targets.

backgroundExploration:
  enabled: true # default: false
  stalenessThresholdSeconds: 600 # default: 600 (10 minutes), minimum: 1
  workerConcurrency: 2 # default: 2, range: 1–16

How it works:

  • Each live request to an alias whose selector is latency, performance, or e2e_performance checks the targets in its active group. Any target whose last probe is older than stalenessThresholdSeconds is enqueued for a background probe.
  • The live request itself is never redirected or delayed — it is served by the selector's best choice.
  • A worker pool (workerConcurrency) drains the queue. Per-target in-flight guards and the global cooldown manager prevent duplicate or unhealthy probes.
  • Each probe is a single canonical chat request (moderate input, two tool definitions, max_tokens: 1000, streaming). It exercises the same transformer + provider path as live traffic, so TTFT / TPS / E2E TPS are measured identically.
  • Probes route via direct/<provider>/<target_model> so they bypass alias resolution and failover, hitting exactly one target.
  • Probes appear in usage records with apiKey = "probe" and attribution = "background" (or "manual" for probes triggered from the management test endpoint). They are real requests and do consume provider quota / cost; budgets and rate caps are out of scope in v1.
  • On cold start, each target's lastProbedAt is initialised to the process start time — the first probe for a target fires at normal cadence once stalenessThresholdSeconds elapses, not eagerly on the first request.

The inline performanceExplorationRate / latencyExplorationRate / e2ePerformanceExplorationRate settings are preserved and continue to take effect when backgroundExploration.enabled is false (the default). All four settings can be edited from the Config page in the admin UI.

Priority Modes

  • selector (default): Selector picks a provider within each group first, then matches API format.
  • api_match: Filter targets to those whose provider supports the incoming API format first, then apply the group's selector. Best for tools requiring specific API features (e.g., Claude Code with Anthropic messages).

Preferred API

For text aliases, when preferred_api is omitted, Plexus advertises a recommended API in GET /v1/models using the canonical target model:

  • Anthropic/Claude models: messages
  • GPT models: responses
  • Gemini models: gemini
  • Everything else: chat_completions

An explicitly configured preferred_api always takes precedence. Non-text aliases do not receive an inferred preferred API.

Prompt Cache Breakpoints

Anthropic prompt caching is prefix caching keyed on cache_control breakpoints. Requests that arrive on /v1/chat/completions or /v1/responses and get translated to the Messages API have no way to express them, so without help every turn pays full input price.

The inject_cache_control behavior fills them in:

Position What it caches
Last tool definition The tools block, which renders first and is the most stable
Last system text block The system prompt, on top of the tools block
Trailing messages (default 2) The conversation tail: current turn end plus previous turn end

Breakpoints are capped at Anthropic's limit of four per request and filled in that order. The trailing marks "walk" with the conversation without any server-side state: Anthropic hashes cumulative prefixes, so a marker that always sits on the current turn advances by itself, and the second one still points at the previous turn's end for turns that emit more than 20 blocks (past the cache lookup window).

It is on wherever the target is Anthropic's own API (api.anthropic.com, Anthropic OAuth, Claude-masking routes) and off everywhere else. Add the behavior explicitly to opt in on Anthropic-compatible upstreams, or with enabled: false to opt out:

models:
  claude-fast:
    advanced:
      - type: inject_cache_control
        ttl: 1h # optional; 5m default, 1h doubles the cache-write cost
        message_breakpoints: 2 # 0 = static prefix only, max 4
        # tools: true
        # system: true

Notes:

  • Only enable it where the target accepts Anthropic cache_control. A wrapped Claude endpoint that passes unknown fields through is fine; one that validates strictly is not.
  • Marked prefixes pay the cache-write premium (1.25x input, 2x with ttl: 1h) on the turn that first writes them. That is a win whenever the prefix is reused across turns and a small loss for one-shot traffic that never reads it back — which is why it only auto-enables on Anthropic's own API. enabled: false opts out.
  • Native Messages clients always keep their own breakpoints, and a translated request that already carries one is left completely untouched.
  • Prompts below the model's minimum cacheable size (512-4096 tokens depending on family) silently don't cache. Nothing breaks, you just get no reads.
  • ttl: 1h also extends the alias's sticky-session window to an hour, so follow-up turns keep landing on the same target.
  • Tool definitions render before the system prompt, so a client that reorders its tool list every turn defeats the cache wherever the breakpoints sit. Plexus does not reorder tools.

Targets

Each target specifies:

  • Provider: Must match an existing provider slug
  • Model: Upstream model name
  • Enabled: Whether this target is active

External Metadata

Model metadata is resolved automatically for every alias and returned by GET /v1/models. Plexus derives a canonical model identity from, in order:

  1. The alias pi_model hint.
  2. The target provider's pi_ai_provider and model's pi_ai_model_id hints.
  3. The enabled target's provider and model names.

It then looks for an exact entry in the configured catalogs. If no exact entry exists, Plexus only infers a display name and input/output modalities from the model name. Context windows, pricing, and supported parameters are never guessed.

For example, this alias needs no metadata configuration:

models:
  claude-sonnet:
    target_groups:
      - name: default
        selector: random
        targets:
          - provider: anthropic
            model: claude-sonnet-4-5

Automatic matching is the default. Use source: auto only when adding overrides:

metadata:
  source: auto
  overrides:
    context_length: 180000
    pricing:
      tiers:
        - input_tokens_above: 272000
          prompt: '0.000005'
          completion: '0.000015'

Pricing tiers are optional. OpenRouter and models.dev tiers are included when the catalog publishes them; configure metadata.overrides.pricing.tiers when you need to supply or replace them for an alias. Each tier applies when the request's input token count exceeds input_tokens_above. Prices use per-token decimal strings, matching the base pricing fields. Omitted prices inherit the base pricing value. /v1/models returns these bands in the alias's pricing.tiers.

To pin an alias to a specific catalog entry instead:

metadata:
  source: models.dev
  source_path: anthropic.claude-sonnet-4-5

To suppress enriched metadata entirely:

metadata:
  source: disabled

Available external catalogs:

Source URL Format
openrouter openrouter.ai provider/model
models.dev models.dev providerid.modelid
catwalk catwalk.charm.sh providerid.modelid

Metadata loads at startup. Failures are non-fatal; automatic resolution falls back to the safe name-based defaults when no exact catalog entry is available.

Direct Model Routing

Bypass aliases entirely using the format direct/<provider>/<model>:

curl ... -d '{"model": "direct/openai_direct/gpt-4o-mini", ...}'
  • Provider and model must exist in configuration
  • Bypasses selector logic and alias settings

Service Tier Suffix

Clients that cannot set service_tier can pick an OpenAI service tier through the model name by appending @<tier> to an alias:

curl ... -d '{"model": "gpt-6-luna@flex", ...}'   # routes as gpt-6-luna with service_tier: flex

Valid tiers are auto, default, flex, priority, and fast (OpenAI treats fast as priority).

  • The suffix is only recognised when the text before it is an alias or an additional_aliases name. An alias that is itself named x@flex is matched as written.
  • It replaces any service_tier in the request body. Provider, model, and alias extraBody are applied afterwards and still win.
  • It is applied to chat and responses targets only; Messages and Gemini targets ignore it. It does not apply to direct/ routing.
  • Key model lists check both names: listing gpt-6-luna allows or excludes all of its tiers, and gpt-6-luna@priority can be listed on its own, for example to keep a key off that tier. Tier entries are matched against the normalised (lower-case) tier, so write them in lower case. priority and fast are one tier, so an entry for either covers both.
  • Usage logs record the name the client sent (gpt-6-luna@flex) as the incoming alias.
  • /v1/models publishes service_tiers using canonical Plexus names (auto, standard, flex, priority, ultrafast) in that order. The list is the union of native capabilities declared per model by enabled targets (pi_ai_quirks.<api>.models[<model>].serviceTierMap or the linked pi-ai catalog model), including nested aliases. Provider-wide serviceTierMap entries are dispatch defaults and are not advertised, since they say nothing about any individual model. A tier only needs support on one target; routing may select another target and rewrite or downgrade it as usual. Unknown capabilities contribute nothing. The field is omitted when no target has known, projectable capabilities; an empty list means the known maps declare no native tiers. Tier pricing is not included, and tier-suffixed model IDs are not added to the model list.

API Keys

API keys authenticate clients to inference endpoints (/v1/*).

Key Settings

Setting Description Required
Name Unique identifier Yes
Secret Bearer token (clients send in Authorization header) Yes
Comment Description or owner No
Quota Name of a quota definition to enforce No

Authentication Methods

Clients can provide credentials via:

  • Authorization: Bearer <secret>
  • Authorization: <secret> (prefix added automatically)
  • x-api-key: <secret>
  • ?key=<secret> query parameter

The /v1/models endpoint is public (no auth required).

Dynamic Attribution

Append :label to track usage without creating separate keys:

Authorization: Bearer sk-plexus-key:copilot
Authorization: Bearer sk-plexus-key:mobile:v2.5

The part before the first colon authenticates; the rest is stored as attribution in usage logs. Query via:

SELECT attribution, COUNT(*), SUM(tokens_input + tokens_output)
FROM request_usage
WHERE api_key = 'key-name'
GROUP BY attribution;

User Quotas

User quotas enforce per-key usage limits. Unlike provider quota checkers (which monitor upstream limits), these control client consumption.

Quota Types

Type Reset Behavior
rolling Continuous window (e.g., "last hour")
daily Resets at UTC midnight
weekly Resets at UTC midnight Sunday
monthly Resets at 00:00 UTC on the 1st

Limit Types

Type What It Counts
requests Number of API calls
tokens Input + output + reasoning + cached tokens
cost Dollar spending (requires pricing on models)

Rolling Window Durations

Supported durations: 30s, 5m, 10m, 30m, 1h, 2h, 2h30m, 6h, 12h, 1d

How Quotas Work

Tokens/Requests (leaky bucket):

  1. After each request, usage is recorded.
  2. Before each request, usage "leaks" based on elapsed time: leaked = elapsed × (limit / duration)
  3. Remaining capacity determines if the request is allowed.

Cost (cumulative):

  1. Spending accumulates as requests complete.
  2. Resets when the window expires.
  3. No leak/refill within the window.

Assigning Quotas

Reference a quota by name in the key's quota field. Keys without a quota have unlimited access.


Cooldowns

When a provider returns errors, Plexus uses an escalating cooldown system to temporarily remove it from the routing pool.

Cooldown Schedule

Consecutive Failures Duration
1st 2 minutes
2nd 4 minutes
3rd 8 minutes
4th 16 minutes
5th 32 minutes
6th 64 minutes
7th 128 minutes
8th 256 minutes
9th+ 300 minutes (cap)

Behavior

  • Successful requests reset failure count to 0.
  • 413 Payload Too Large errors do NOT trigger cooldowns (client error).
  • Each provider+model combination tracks failures independently.
  • Cooldowns persist in the database across restarts.

Configuration

Setting Description Default
initialMinutes First failure duration 2
maxMinutes Cap for exponential backoff 300

Disabling Per Provider

Set disable_cooldown: true on a provider to exclude it from the cooldown system. Recommended for:

  • Local model servers (Ollama, LM Studio)
  • Providers with their own rate-limit handling
  • Testing scenarios

Do not disable for cloud providers with unreliable endpoints.

Disabling cooldowns removes the provider's existing cooldowns. Quota checks continue to report usage but do not create cooldowns until cooldowns are enabled again.

Management API

  • GET /v0/management/cooldowns — list active cooldowns
  • DELETE /v0/management/cooldowns — clear all
  • DELETE /v0/management/cooldowns/:provider?model=:model — clear specific

See API Reference: Cooldown Management.


Request Timeouts

Plexus can abort upstream requests that run too long instead of waiting forever.

Timeouts matter for both reliability and cost:

  • they stop abandoned or hung upstream requests from continuing to burn quota,
  • they allow the dispatcher to fail over to another provider when appropriate,
  • and they put a hard cap on runaway or extremely slow streams.

Default Behavior

  • Global default timeout: 300 seconds
  • Per-provider override: optional via timeoutMs
  • Effective timeout: provider.timeoutMs ?? (global timeout.defaultSeconds × 1000)

If the timeout fires before the request completes:

  • Plexus aborts the upstream fetch,
  • the request is recorded with responseStatus = "timeout",
  • and failover may continue to the next provider when the dispatcher is still in a retryable/failover-safe stage.

Global Timeout Configuration

Global timeout settings live in system settings and are exposed through the Admin UI Config → Timeout Settings and the management API.

Field Type Default Range Meaning
defaultSeconds integer 300 1–3600 Default maximum duration for any upstream request unless the selected provider overrides it

Per-Provider Timeout Configuration

Field Type Default Meaning
timeoutMs integer (milliseconds) inherit global Override the global timeout for requests routed to this provider

Management API

  • GET /v0/management/config/timeout — returns the effective global timeout config
  • PATCH /v0/management/config/timeout — partial update of timeout settings

Example:

PATCH /v0/management/config/timeout
{
  "defaultSeconds": 120
}

Tuning Guidance

  • Start with the default 300s if you are unsure.
  • Lower it for providers that should either respond quickly or fail fast.
  • Raise it only for providers that legitimately need long-running responses.
  • Prefer a provider-specific override over increasing the global timeout for everyone.

Stall Detection

Stall detection protects against providers that are technically connected but behaving too slowly to be useful.

Unlike a plain timeout, stall detection can distinguish between:

  • a provider that never starts producing meaningful bytes (TTFB stall), and
  • a provider that starts streaming but then slows below an acceptable throughput floor (throughput stall).

By default, stall detection is effectively off because both threshold dimensions are disabled:

  • ttfbSeconds = null
  • minBytesPerSecond = null

The supporting tuning values still have defaults even when detection is disabled:

  • ttfbBytes = 100
  • windowSeconds = 10
  • gracePeriodSeconds = 30

Two Stall Dimensions

1. TTFB Stall Detection

TTFB (time-to-first-bytes) detection checks whether the provider produces enough bytes quickly enough.

It uses two values together:

  • ttfbSeconds — how long Plexus waits
  • ttfbBytes — how many bytes must arrive within that time to count as meaningful output

If the threshold is not met in time, Plexus treats the provider as stalled and aborts that attempt.

Important: when this happens before any response bytes reach the client, the dispatcher can transparently fail over to another provider.

2. Throughput Stall Detection

Throughput detection applies after the provider has started responding.

It uses:

  • minBytesPerSecond — minimum acceptable streaming rate
  • windowSeconds — sliding window width for measuring throughput
  • gracePeriodSeconds — delay after TTFB success before throughput enforcement starts

The grace period is especially important for reasoning-heavy models that may pause naturally after their first chunk.

If throughput drops below the configured floor, Plexus aborts the stream and records the request as stall.

Important: once bytes have already been sent to the client, Plexus cannot transparently fail over the same response stream. The client must retry.

Global Stall Settings

Global stall settings live in the Admin UI under Config → Stall Detection and in the management API.

Field Type Default Range Meaning
ttfbSeconds integer or null null 5–120 or null Max time to wait for the first meaningful bytes. null disables TTFB stall detection.
ttfbBytes integer 100 50–10000 Byte threshold that must arrive within ttfbSeconds to count as “started”.
minBytesPerSecond integer or null null 50–5000 or null Minimum acceptable streaming throughput. null disables throughput stall detection.
windowSeconds integer 10 3–30 Sliding window width used to calculate throughput.
gracePeriodSeconds integer 30 0–120 Delay after TTFB success before throughput enforcement starts.

Effective Behavior and Inheritance

  • Stall detection is enabled for a request if either ttfbSeconds or minBytesPerSecond is active after global + provider override resolution.
  • Provider overrides take precedence over global settings.
  • Per-provider overrides can enable stall detection even if the global stall config is disabled.
  • Leaving provider fields empty keeps the global value for that field.

Management API

  • GET /v0/management/config/stall — returns the current global stall detection config
  • PATCH /v0/management/config/stall — partial update of stall settings

Example:

PATCH /v0/management/config/stall
{
  "ttfbSeconds": 15,
  "ttfbBytes": 100,
  "minBytesPerSecond": 500,
  "windowSeconds": 10,
  "gracePeriodSeconds": 30
}

Disable one dimension while keeping the other:

PATCH /v0/management/config/stall
{
  "ttfbSeconds": null,
  "minBytesPerSecond": 400
}

Recommended Starting Points

  • Fail over slow starters only: set ttfbSeconds, leave minBytesPerSecond as null
  • Protect long streams too: set both ttfbSeconds and minBytesPerSecond
  • Reasoning-heavy models: keep a longer gracePeriodSeconds
  • Bursty but healthy streams: increase windowSeconds before raising minBytesPerSecond

Relationship to Client Disconnects

Separate from timeout/stall settings, Plexus now also cancels upstream provider requests when the downstream client disconnects during streaming. This reduces wasted quota for abandoned requests even when neither timeouts nor stall detection are enabled.


MCP Servers

Plexus proxies Model Context Protocol servers. Only HTTP streaming transport is supported.

Settings

Setting Description Required
Server Name Identifier used in URLs Yes
Upstream URL Full MCP server endpoint Yes
Enabled Active for routing No
Headers Static headers forwarded to upstream No

Endpoints

Method Path Description
POST /mcp/:name JSON-RPC messages
GET /mcp/:name SSE streaming
DELETE /mcp/:name End session

Authentication

All MCP endpoints require a Plexus API key. Client auth headers are NOT forwarded — only configured static headers are added.

OAuth Discovery

Plexus exposes standard OAuth 2.0 endpoints for MCP clients:

  • GET /.well-known/oauth-authorization-server
  • GET /.well-known/oauth-protected-resource/mcp/:name
  • GET /oauth/authorize and POST /oauth/authorize
  • POST /oauth/token
  • POST /oauth/register

The authorization server is shared by all configured MCP servers, while each /mcp/:name endpoint has its own protected-resource metadata and RFC 8707 resource identifier (<issuer>/mcp/:name).

The browser consent step uses the existing Plexus session. A limited API-key session is always bound to that key and cannot select another key. An administrator session must explicitly choose the API-key identity to which the MCP grant will be bound, even when only one active key exists.


Vision Fallthrough

Vision fallthrough allows image inputs to be preprocessed by a vision-capable model before routing to the actual target.

Configuration:

  1. Set a global Descriptor Model in Settings (Admin UI)
  2. Enable Use Image Fallthrough on individual model aliases

Images are sent to the descriptor model first; the text analysis is prepended to the original prompt.


Pricing

Configure pricing to enable cost selector strategy, cost-based quotas, and usage reporting.

Pricing Sources

Source Description
simple Fixed per-million token rates
openrouter Live rates from OpenRouter API
defined Tiered rates based on input token volume
per_request Flat fee per API call

Simple Pricing

Configure via Admin UI or API:

  • input: dollars per million input tokens (e.g., 3.00)
  • output: dollars per million output tokens (e.g., 15.00)
  • cached: cache read rate (optional)
  • cache_write: cache write rate (optional)

OpenRouter Pricing

Fetches live rates from OpenRouter. Configure via Admin UI or API:

  • Set source to openrouter
  • Set model slug (e.g., anthropic/claude-3.5-sonnet)
  • Optional discount for percentage off all rates

Tiered Pricing

Useful for providers with volume discounts. Configure tiers via Admin UI or API:

Lower Bound Upper Bound Input Rate Output Rate
0 200,000 $3.00/M $15.00/M
200,001 ∞ (infinity) $1.50/M $7.50/M

Per-Request Pricing

Flat fee regardless of token count. Configure via Admin UI or API:

  • Set source to per_request
  • Set amount (e.g., 0.04)

Full cost stored in costInput; output/cached fields are zero.


Token Estimation

Some providers (especially free-tier models) don't return usage data. Enable token estimation to automatically calculate token counts using a character-based heuristic.

Enable via:

  • Provider setting: estimateTokens: true
  • Admin UI: Advanced → Estimate Tokens

Estimated counts are flagged with tokensEstimated = 1 in usage records. Typical accuracy is within ±15% of actual values.


Encryption at Rest

Plexus can encrypt sensitive data using AES-256-GCM.

What Gets Encrypted

Data Fields
API Keys secret
OAuth Credentials accessToken, refreshToken
Providers apiKey, headers, quotaCheckerOptions
MCP Servers headers

Setup

# Generate a key
openssl rand -hex 32

# Set environment variable
export ENCRYPTION_KEY="your-64-character-hex-key"

Key format accepts:

  • 64-character hex string (32 bytes, used directly)
  • Arbitrary passphrase (derived via scrypt)

Behavior

  • Existing plaintext data encrypts on first startup with key set.
  • New data encrypts on write, decrypts on read.
  • API key authentication uses SHA-256 hash lookups.

Key Rotation

# Docker
docker exec -e ENCRYPTION_KEY="old" -e NEW_ENCRYPTION_KEY="new" plexus ./plexus rekey

# Binary
ENCRYPTION_KEY="old" NEW_ENCRYPTION_KEY="new" ./plexus rekey

After re-keying, update ENCRYPTION_KEY before restarting.

Important

  • Lost keys = unreachable data. Back up keys securely.
  • Encrypted values prefixed with enc:v1: in database.
  • Without ENCRYPTION_KEY, all data stored plaintext.

Failover

Plexus automatically retries failed requests across alternative targets in multi-target model aliases.

Default Behavior

All non-2xx status codes except 400 and 422 trigger failover. Custom retryable codes and errors can be configured.

Configuration Options

  • enabled: Toggle failover on/off
  • retryableStatusCodes: List of status codes that trigger retry
  • retryableErrors: List of network errors that trigger retry