Skip to content

Refactor configuration: replace endpoint and responsesEndpoint with new interfaces config #1615

Description

@khshanovskyi

Name and Version

0.45

What is the problem this feature will solve?

Support for multiple AI API (OpenAI, Anthropic, Gemini) at the Core level.

What is the feature you are proposing to solve the problem?

Extend the deployment config with a new interfaces field.

{
  "models": {
    "gpt-5.5": {
      "displayName": "GPT 5.5",
      "type": "chat",
      "endpoint": "http://localhost:6001/openai/deployments/gpt-5.2/chat/completions", // will be deprecated
      "responsesEndpoint": "http://localhost:6001/openai/v1/responses", // will be deprecated
      "interfaces": {
        "openaiChatCompletions": {
          "base_url": "http://adapter-dial-openai:5000/"
        },
        "openaiResponses": {
          "base_url": "http://adapter-dial-openai:5000/"
        },
        "anthropicMessages": {
          "base_url": "http://adapter-dial-bedrock:5000/to-responses"
        },
        "geminiGenerateContent": {
          "base_url": "http://adapter-dial-vertexai:5000/to-responses"
        }
      },
      "upstreams": []
    }
  }
}

Interfaces will consist of:

  1. map with constant types that describe vendor API:
  • openaiChatCompletions — the OpenAI "deployments" POST family (chat/completions, completions, embeddings)
  • openaiResponses — the OpenAI Responses API
  • anthropicMessages — (will be added later to support Anthropic Messages API)
  • geminiGenerateContent — (will be added later to support Gemini generateContent API)
  • geminiInteractions — (will be added later to support Gemini Interactions API)
  1. Each object will have the base_url to access model

The flow will be:

  1. Get a request to {core-host}/openai/v1/responses
  2. Parse body → read model (deployment id)
  3. Resolve deployment
  4. Capability gate , check if deployment supports such an endpoint
  5. Pick upstream from deployment config
  6. Build URI -> {interfaces.openaiResposes.base_url} + /openai/v1/responses
  7. Proxy & stream back (rewrite response id).

An additional task is to support auto-migration for endpoint and responsesEndpoint. It can be done via configuring an additional env variable ENDPOINT_MIGRATION_TO_INTERFACES, when enabled, migration will be performed in the source file; when disabled (default) will migrate to interfaces only in memory

What alternatives have you considered?

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

Status
No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions