Summary
shouldUseResponsesAPI (bifrost/bifrost.go, ~L1589 on v2.4.2) forces the Responses API whenever the provider supports native tools AND any native tool is enabled -- overriding an explicit operator useResponsesAPI: false. Since agent creation defaults web_search into enabledNativeTools, a freshly created agent on an "OpenAI Compatible" service routes to /v1/responses even though the operator disabled the Responses API, and even though the service points at a backend (llama.cpp, Ollama, vLLM) that cannot serve it.
The resulting failure is hard 400s surfaced to users as "Sorry! An error occurred while accessing the LLM" with a backend-specific parse error in the logs (llama.cpp: Cannot determine type of 'item' once conversation history is included) -- nothing points at native tools as the cause. Diagnosing this required reading the plugin source; the fix was discovering the hidden default and clearing enablednativetools in the DB.
Steps to reproduce
- "OpenAI Compatible" service -> llama.cpp backend, "Use Responses API" = false.
- Create an agent bound to that service (creation default gives it native
web_search).
- Mention the agent. Backend receives
POST /v1/responses (observed in its request log) despite the toggle; llama.cpp 400s on follow-up messages.
Expected
An explicit operator useResponsesAPI: false wins -- native tools that require the Responses API should be filtered out for services that have it disabled (with a log/UI warning), not silently flip the transport. filterNativeToolsForServiceType looks like the natural place.
Related
The ChatOnly fallback registration comment ("a local Ollama/vLLM server ... must be registered chat-only so Bifrost downgrades Responses-API requests instead of failing") shows the downgrade machinery exists -- it just is not applied to the primary service in this path.
Env: plugin 2.4.2, server 11.9.0, llama.cpp backend. Happy to PR given maintainer direction on the preferred behavior.
🤖 Generated with Claude Code
https://claude.ai/code/session_016e12GP7EHU2KeCGGYcfvBo
Summary
shouldUseResponsesAPI(bifrost/bifrost.go, ~L1589 on v2.4.2) forces the Responses API whenever the provider supports native tools AND any native tool is enabled -- overriding an explicit operatoruseResponsesAPI: false. Since agent creation defaultsweb_searchintoenabledNativeTools, a freshly created agent on an "OpenAI Compatible" service routes to/v1/responseseven though the operator disabled the Responses API, and even though the service points at a backend (llama.cpp, Ollama, vLLM) that cannot serve it.The resulting failure is hard 400s surfaced to users as "Sorry! An error occurred while accessing the LLM" with a backend-specific parse error in the logs (llama.cpp:
Cannot determine type of 'item'once conversation history is included) -- nothing points at native tools as the cause. Diagnosing this required reading the plugin source; the fix was discovering the hidden default and clearingenablednativetoolsin the DB.Steps to reproduce
web_search).POST /v1/responses(observed in its request log) despite the toggle; llama.cpp 400s on follow-up messages.Expected
An explicit operator
useResponsesAPI: falsewins -- native tools that require the Responses API should be filtered out for services that have it disabled (with a log/UI warning), not silently flip the transport.filterNativeToolsForServiceTypelooks like the natural place.Related
The
ChatOnlyfallback registration comment ("a local Ollama/vLLM server ... must be registered chat-only so Bifrost downgrades Responses-API requests instead of failing") shows the downgrade machinery exists -- it just is not applied to the primary service in this path.Env: plugin 2.4.2, server 11.9.0, llama.cpp backend. Happy to PR given maintainer direction on the preferred behavior.
🤖 Generated with Claude Code
https://claude.ai/code/session_016e12GP7EHU2KeCGGYcfvBo