Skip to content

README.md

Latest commit

 

History

History
75 lines (55 loc) · 2.63 KB

README.md

File metadata and controls

75 lines (55 loc) · 2.63 KB

Architecture

Overview

The Plugins Adapter is currently implemented as an Envoy external processor (ext-proc) that provides a plugin framework for intercepting and processing Envoy messages.

Plugin Adapter components

External Processor Server

  • gRPC server implementing Envoy's external processor interface
  • Intercepts HTTP requests/responses at various stages
  • Invokes plugin manager with the appropriate hooks

Plugin Manager

  • Leveraged from the ContextForge project
  • Loads and initializes plugins from configuration
  • Manages plugin lifecycle
  • Coordinates hook invocations across plugins

Current hook points include but are not limited to:

  • Tool Pre-Invoke: Before tool execution
  • Tool Post-Invoke: After tool returns results
  • Prompt Pre-Fetch: Before prompt retrieval

Plugins

Plugins can be implemented and deployed in two ways:

Internal Plugins

  • Deployed within the same container/pod as the plugin manager
  • Loaded directly by the plugin manager
  • Lower latency, simpler deployment
  • Examples: Built-in validation, filtering, and transformation plugins

External Plugins

  • Deployed as separate services
  • Communicate via API conforming to the expected plugin interface (currently MCP)
  • Greater isolation and independent scaling
  • Can be written in any language
  • Examples: External policy engines (OPA, Cedar), ML-based content moderation service

Example Tool Request Flow

The MCP gateway router is included to show the relationship with the ext-proc.

graph TD
    A[Client] -->|1. MCP Tool Call Request| B[Envoy Gateway]
    B -->|2. Route Request| C[MCP Gateway Router ext-proc]
    C -->|Route Decision| B
    B -->|3. Forward| D[Plugins Adapter ext-proc]
    D -->|4. Parse MCP Message| E[Plugin Manager]
    E -->|5. Invoke Hooks| F[Plugins]
    F -->|6. Process/Validate/Modify| E
    E -->|7. Response| D
    D -->|7. Response| B
    B -->|8a. Forward to Upstream| G[MCP Tool Server]
    B -->|8b. Or Return Error| A
    G -->|9. Response*| B
    B -->|10. Response| A

    style D fill:#bbf,color:#000000
    style E fill:#bbf,color:#000000
    style F fill:#bbf,color:#000000
Loading

*Note: Tool server responses (step 9) also flow through the Plugins Adapter ext-proc for validation/modification before returning to client. This is not explicitly illustrated in this tool request flow diagram.

System Architecture

Plugin adapter example with plugins

System Architecture