[Proposal] Support for MCP protocol

[yanavlasov]

The Model Context Protocol (MCP) is an open, stateless/stateful protocol that allows GenAI applications to retrieve and exchange context (e.g. source code, files, documents) with LLMs, using JSON-RPC semantics. For more context see envoyproxy/envoy#39174

Adding MCP support to ai-gateway would provide a more comprehensive "package" for serving GenAI requests.

At a minimum ai-gateway could provide the following functionality:

1. Routing policy to backends with retries.
2. Authentication of requests
3. Authorization per RPC method
4. Access logging of RPCs
5. Ratelimiting per RPC
6. Transcoding JSON-RPC messages to existing API surfaces, for example gRPC and OpenAPI.

Going forward it could:

1. Provide external callouts to inspect MCP requests and responses, for example to provide additional filtering or custom policy.
2. Multiplexing/demultiplexing of batches.

[vladfr]

Seconded. I am working on a similar project as I think MCP servers will be widely deployed in the coming months, as StreamableHTTP becomes available.

[missBerg]

Agree. I wonder if we should consider something like MCPGatewayRoute to support this idea, and look forward to seeing the work on the dataplane side in envoy proxy.

[daixiang0]

Any plan about this?

[missBerg]

@daixiang0 yes! there are a few conversations ongoing :) so feel free to chime in with any thoughts you have and requirements. Also check out #775

[ion-elgreco]

Could also make sense to integrate into https://agentgateway.dev/? This what they have done at kgateway