[Enhancement]: custom endpoint implementation with openai responses api not compatible with anthropic native web search

[xinguaws]

What features would you like to see added?

this is my custom endpoint definition, it's using claude models via google's vertex ai through portkey gateway

  custom:
    - name: "vertex claude"
      apiKey: "dummy"
      baseURL: "https://<portkey-gw>/v1"
      headers:
        x-portkey-api-key: "${portkey_api_key}"
        x-portkey-provider: "@vertex-ai"
        x-portkey-metadata: '{"_user": "{{LIBRECHAT_USER_EMAIL}}"}'
      models:
        default: [
          "anthropic.claude-sonnet-4-6"
        ]
        fetch: false
      customParams:
        defaultParamsEndpoint: 'anthropic'
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "claude via vertex"   

I found that native web search tool not working so I did my own research, looks like librechat's custom endpoint is using openai responses api, logs i captured looks like below

  "body": {
    "model": "anthropic.claude-sonnet-4-6",
    "stream": true,
    "tools": [
      {
        "type": "web_search_20250305",
        "name": "web_search"
      }
    ],
    "text": {},
    "max_output_tokens": 64000,
    "metadata": {
      "user_id": ""
    },
    "thinking": {
      "type": "adaptive"
    },
    "promptCache": true,
    "input": [
      {
        "type": "message",
        "role": "user",
        "content": "can you search the web and let me the know the weather today in dallas tx?"
      }
    ]
  }

If I use anthropic endpoint with google vertex ai as provider, it works, the logs look like this

  "body": {
    "max_tokens": 1000,
    "messages": [
      {
        "role": "user",
        "content": "What is the weather today in Dallas, TX?"
      }
    ],
    "model": "anthropic.claude-sonnet-4-6",
    "tools": [
      {
        "type": "web_search_20250305",
        "name": "web_search"
      }
    ]
  }

More details

from above logs capturing and comparison, I think this is librechat's custom endpoint SDK issue, to invoke anthropic native web search tool, it expects messages instead inputs, can we use anthropic SDK instead of openai sdk
when the custom endpoint is anthropic?

      customParams:
        defaultParamsEndpoint: 'anthropic'

for example, using anthropic sdk would work like this

client = Anthropic(
    base_url=base_url,
    api_key='dummy',
    default_headers={
        "x-portkey-api-key": api_key
    },
)

response = client.messages.create(
    model=model,
    max_tokens=1000,
    tools=[{"type": "web_search_20250305", "name": "web_search"}],
    messages=[
        {
            "role": "user",
            "content": "What is the weather today in Dallas, TX?"
        }
    ]
)

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[xinguaws]

@danny-avila is it not a valid feature request? let me know if this is already working but somehow I misconfigured it, also I forgot to mention, custom endpoint with google gemini models also not working properly for "Grounding with Google Search" feature, I am not sure if this is a similar issue

[xinguaws]

ok, this was indeed already implemented, web search is working for me now using 0.8.7-rc1 version with provider: 'anthropic' in the custom endpoint definition, @danny-avila do you have a plan to implement provider: 'gemini' for custom endpoint?

[danny-avila]

Glad that sorted it, and yes, you diagnosed it correctly. provider: 'anthropic' is what switches the transport to the native Anthropic /v1/messages client. defaultParamsEndpoint on its own only changes the parameter schema (which fields and tools the UI builds), not the wire format, so the request was still going out through the OpenAI-compatible client and flipping to the Responses API once web search was enabled. Setting provider is what makes the endpoint speak Anthropic natively.

The Gemini case is the same root cause: without a native provider, a custom endpoint falls back to the OpenAI-compatible client, so "Grounding with Google Search" can't be expressed since it's a native Google tool (the equivalent of Anthropic's web_search_20250305). And provider currently only accepts anthropic.

Yes, extending it to Google/Gemini is on the roadmap. The plan is to support the native Google provider (Vertex + Gemini) the same way, so grounding works through custom endpoints and gateways like Portkey. I'll follow up here when it lands.

[xinguaws]

@danny-avila awesome, thank you very much 🚀