Skip to content

tools-and-mcp.mdx

Latest commit

 

History

History
189 lines (134 loc) · 7.79 KB

tools-and-mcp.mdx

File metadata and controls

189 lines (134 loc) · 7.79 KB
title Tools and MCP
description Unified tool registry, MCP server integration, tool format conversion, and context window overhead for tool-equipped requests in FrugalRoute.

Tools and MCP

FrugalRoute manages both models and their tool ecosystems. Tools from MCP servers, OpenAI function calling, Anthropic tool use, and custom local handlers are normalized into a unified registry so that any model can use any tool regardless of its origin or native format.

Tool Registry

The tool registry is the central data structure that tracks every available tool. Each tool entry stores a canonical representation (name, description, input schema) plus metadata about where the tool came from and how to execute it.

Tools enter the registry from three sources:

MCP Tools

Discovered automatically from connected MCP servers via the tools/list JSON-RPC method. When an MCP server connects, the router fetches its tool manifest and registers each tool with the server's identifier as the origin. If the server's tool list changes, the registry is updated on the next tools/list call.

Provider Tools

Passed through to model adapters in the provider's native format. These are tools defined by the caller in an OpenAI functions array or an Anthropic tools array. The registry normalizes them into the internal schema so they can be matched and routed, then converts them back to the provider's format when forwarding the request.

Custom Tools

Locally registered handler functions. A custom tool is a name, description, JSON Schema for its parameters, and a handler function that receives the parsed arguments and returns a result. Custom tools are registered in code and execute entirely within the router process.

registry.registerCustomTool({
  name: "lookup_user",
  description: "Look up a user by email address",
  inputSchema: {
    type: "object",
    properties: {
      email: { type: "string", description: "The user's email address" },
    },
    required: ["email"],
  },
  handler: async (args) => {
    const user = await db.query("SELECT * FROM users WHERE email = ?", [args.email]);
    return { content: [{ type: "text", text: JSON.stringify(user) }] };
  },
});

MCP Server Integration

FrugalRoute connects to MCP servers as a client, discovering their tools and forwarding tool calls during multi-turn conversations.

Transports

Three transport types are supported:

Transport Use Case Configuration
stdio Local processes (e.g. a CLI tool wrapped as an MCP server). The router spawns the process and communicates over stdin/stdout. command and optional args, env fields.
sse Remote servers using Server-Sent Events over HTTP. url field pointing to the SSE endpoint.
streamable-http Remote servers using the Streamable HTTP transport (bidirectional over a single HTTP connection). url field pointing to the HTTP endpoint.

Configuration

MCP servers are declared in the router configuration:

mcpServers:
  - id: filesystem
    transport: stdio
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/data"]

  - id: remote-tools
    transport: streamable-http
    url: https://tools.example.com/mcp
    apiKey: ${REMOTE_TOOLS_API_KEY}

The optional apiKey field is sent as a Bearer token in the Authorization header for SSE and Streamable HTTP transports.

Auto-Discovery

On startup (and on reconnection), the router sends a tools/list JSON-RPC request to each configured server. The returned tools are merged into the unified registry. If a server becomes unavailable, its tools are marked as offline and excluded from routing until the server reconnects.

Tool Execution

When a model response includes one or more tool calls, the router resolves each call against the registry and dispatches it to the correct executor.

Dispatch Rules

Tool Source Execution Path
MCP tool Forwarded to the originating MCP server via a tools/call JSON-RPC request. The server executes the tool and returns the result.
Custom tool Executed locally by invoking the registered handler function.
Provider tool Returned to the caller as-is (the caller is responsible for execution and feeding results back).

Batch Execution

When a model emits multiple tool calls in a single response, the router executes all independent calls in parallel using Promise.all. This reduces round-trip latency for models that support parallel tool use.

Error Handling

Tool execution errors are returned as structured results with isError: true rather than throwing exceptions. This follows the MCP convention and ensures that the model receives the error as context for its next generation step instead of crashing the request.

// Successful result
{ content: [{ type: "text", text: "..." }], isError: false }

// Failed result
{ content: [{ type: "text", text: "Connection refused" }], isError: true }

Format Conversion

Models from different providers expect tools in different wire formats. The router converts automatically between all supported formats so that any tool can be presented to any model.

OpenAI Function Calling Format

{
  "type": "function",
  "function": {
    "name": "lookup_user",
    "description": "Look up a user by email address",
    "parameters": {
      "type": "object",
      "properties": {
        "email": { "type": "string", "description": "The user's email address" }
      },
      "required": ["email"]
    }
  }
}

Anthropic Tool Format

{
  "name": "lookup_user",
  "description": "Look up a user by email address",
  "input_schema": {
    "type": "object",
    "properties": {
      "email": { "type": "string", "description": "The user's email address" }
    },
    "required": ["email"]
  }
}

Internal MCP Format

The registry stores tools in the MCP canonical form (name, description, inputSchema). Conversion to and from provider formats happens at the adapter boundary, so the rest of the router only deals with one representation.

Conversion Matrix

From To Notes
MCP OpenAI inputSchema mapped to function.parameters.
MCP Anthropic inputSchema mapped to input_schema.
OpenAI MCP function.parameters mapped to inputSchema.
Anthropic MCP input_schema mapped to inputSchema.
OpenAI Anthropic Converted through MCP as intermediate.
Anthropic OpenAI Converted through MCP as intermediate.

Context Window Management

Tool definitions are included in the prompt sent to the model, which means they consume context window tokens. The router accounts for this overhead when checking whether a request fits within a model's context limit.

Token Overhead

A typical tool definition costs approximately 100 tokens depending on the length of its description and the complexity of its parameter schema. The router estimates tool token usage by running each tool's JSON representation through the same token counter used for messages.

How It Affects Routing

When the router evaluates whether a model can handle a request, it calculates:

available = contextWindow - toolTokens - systemPromptTokens - messageTokens - reservedOutputTokens

If available is negative, the model is skipped in favor of one with a larger context window, or the tool set is pruned to fit. This prevents silent truncation of tool definitions that would cause the model to hallucinate tool schemas or miss available tools entirely.

Reducing Overhead

For requests that include a large number of tools, consider:

  • Scoping MCP server connections to only the servers relevant to the current task.
  • Using shorter tool descriptions when the model does not need extensive guidance.
  • Setting a maxTools limit in the route configuration to cap the number of tools injected per request.