Support Native Anthropic API

[xiaolin593]

Description:
Support native Anthropic API so that user can enable Claude Code via Anthropic as some features are not available in OpenAI schema.

It's evident that Anthropic API is becoming an unified API for Claude model as cloud providers both GCP and AWS support Anthropic API.

• Anthropic Native API
  • Support native Anthropic API v1/messages for first party Anthropic
  • Support native Anthropic API v1/messages for GCP Anthropic
  • Support native Anthropic API v1/messages for AWS Anthropic

Relevant Links:

• Anthropic Claude Code

• AWS Anthropic Claude Message

• GCP Claude Model

[yuzisun]

Claude code SDK supports first party Anthropic API and third party AWS bedrock, GCP Vertex. Currently AIGW only supports OpenAI API, however there is no reason to translate back and forth from Anthropic to OpenAI and then back to Anthropic to access Claude models. Their API structures and conversational paradigms for handling tools, thinking differ, reflecting their underlying design philosophies, so it is beneficial for Envoy AI Gateway to support Anthropic API natively to access Claude models.

[mathetake]

Am I understanding correctly that this is about supporting Anthropic API as an input schema?

[yuzisun]

Am I understanding correctly that this is about supporting Anthropic API as an input schema?

Correct, so user can use anthropic SDK or Claude code which is Anthropic's agentic coding tool directly to access Claude family models via Envoy AI Gateway.

[mathetake]

sounds good

[yuzisun]

Anthropic offers a rich set of advanced features—such as tool use, prompt caching, extended context (“thinking”), batch processing, and PDF handling—across its first-party API, as well as integrations with AWS Bedrock and Google Cloud Vertex AI (docs). Supporting the native Anthropic API directly enables full access to these capabilities without the limitations introduced by protocol translation. Since multiple providers are supported, fallback can still work using the Anthropic API. Also see litellm's pass through endpoints https://docs.litellm.ai/docs/pass_through/intro.

Proposal

To support Anthropic API natively we can introduce anthropic/v1/messages endpoint on Envoy AI Gateway for main stream model provider APIs without protocol translation to differentiate from the unified API(with protocol translation). In order to make anthropic SDK compatible, ANTHROPIC_BASE_URL can be set to https://api.anthropic.com/anthropic.

server.Register("anthropic/v1/messages",extproc.AnthropicMessageProcessorFactory(MessageMetrics))

Implement Anthropic Message Processor which does two parts:
Router filter to extract the model name to the HTTP header x-ai-eg-model for matching the routing rule to the correct model provider backends. Upstream filter for translation to the model provider protocol:

• This requires very minimal translation as both GCP Vertex AI and AWS bedrock offer the native Anthropic API.
• AWS Bedrock supports passing the anthropic message payload using InvokeModel and InvokeModelWithResponseStream , see https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages-request-response.html.
  • The url path needs to be mutated from v1/messages to bedrock path format with the model name.
    The anthropic_version is passed in the request body(rather than header) and must be set to the value like bedrock-2023-10-16.
  • GCP Vertex API for accessing Claude is nearly-identical to the Message API and supports all of the same options with two key differences
    In Vertex AI, the model name is not passed in the request body, instead it is specified in google cloud endpoint url. The upstream filter needs to mutate the url path with the model name.
    In Vertex AI, the anthropic_version is passed in the request body(rather than header) and must be set to the value like vertex-2023-10-16.

[missBerg]

Should we tag this for 0.4 @yuzisun?

[mathetake]

added to v0.4 milestone