Skip to content

Add Transparent HTTP Proxy Mode for Native Bedrock API Access #170

@fabianx-ai

Description

@fabianx-ai

Is your feature request related to a problem? Please describe.

Currently, the Bedrock Access Gateway requires complex request/response parsing and format conversion between different
API styles (OpenAI, Anthropic, native Bedrock). This approach has several limitations:

  • Format drift risk: Custom parsing can become outdated when AWS updates their APIs
  • Maintenance overhead: Need to manually implement every new Bedrock feature and parameter
  • Performance impact: JSON parsing and reconstruction adds latency
  • Limited compatibility: Headers, query parameters, and edge cases may not be handled correctly
  • Development complexity: Each new Bedrock endpoint requires custom implementation

Describe the feature you'd like

Add a transparent HTTP proxy mode for direct Bedrock API access that:

  1. Universal endpoint: Single route /bedrock/model/{model_id}/{endpoint_path:path} that handles ALL Bedrock
    endpoints automatically
  2. Pure passthrough: Forward requests to AWS Bedrock as-is with zero parsing or modification
  3. Authentication swap: Replace proxy API key with AWS bearer token transparently
  4. Header preservation: Forward all original headers (User-Agent, custom headers, etc.)
  5. Future-proof: New AWS Bedrock features work immediately without code changes
  6. Method agnostic: Support GET, POST, PUT, DELETE, PATCH automatically
  7. Streaming support: Detect and handle streaming responses automatically

Implementation approach:

  • Environment variable AWS_BEARER_TOKEN_BEDROCK enables transparent proxy mode
  • URL rewriting: localhost/api/v1/bedrock/model/X/invokebedrock-runtime.{region}.amazonaws.com/model/X/invoke
  • Client uses same proxy API key, backend uses AWS token (transparent to client)
  • Use async HTTP client (httpx) for optimal performance

Additional context

This feature would provide:

  • Perfect AWS compatibility - Always matches official AWS API exactly
  • Zero maintenance overhead - New AWS features work automatically
  • Better performance - No JSON parsing/reconstruction overhead
  • Broader compatibility - Handles all HTTP methods, headers, query params
  • Future-proof design - Works with any future Bedrock API changes

Example usage remains the same for clients:

curl -H "Authorization: Bearer proxy-api-key" \
  -H "Content-Type: application/json" \
  -d '{"messages": [...]}' \
  "http://localhost:8000/api/v1/bedrock/model/claude-3/invoke"

But internally it becomes a simple HTTP proxy instead of complex format conversion.

This would complement the existing OpenAI-compatible endpoints while providing a "native" path for maximum AWS Bedrock
compatibility.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions