fix(acp): deliver plain replies for openai-compat/Ollama agents (rebased) - #1
Draft
imp0stor wants to merge 4 commits into
Draft
fix(acp): deliver plain replies for openai-compat/Ollama agents (rebased)#1imp0stor wants to merge 4 commits into
imp0stor wants to merge 4 commits into
Conversation
buzz-agent's contract is that output is tool calls: streamed assistant text
is observability only and never published. Capable models honour that and
call `send_message`. Small local models served over shared compute do not
always — on multi-step turns they run their tools, then write the answer as
prose and end the turn, so the turn reports success with nothing in the
channel.
Retain the turn's streamed text in AcpClient (bounded at 8 KiB) alongside a
flag for whether a publish tool call was seen, and on EndTurn publish that
text as a threaded channel reply when the agent published nothing itself.
Gated on EndTurn only: MaxTokens / MaxTurnRequests mean the turn was
truncated, so the text is not a deliberate answer. Bare acknowledgements
("OK", "Done") are filtered, and taking the buffer clears it so the same
text can never post twice.
Off by default via --deliver-plain-replies / BUZZ_ACP_DELIVER_PLAIN_REPLIES:
for cloud models, which reliably publish their own replies, posting streamed
text would double-post. The desktop shared-compute preset opts in, so this
stays scoped to the case that needs it.
Verified end to end against a live relay with stub ACP agents that pin each
branch deterministically: a prose-only agent gets its answer delivered, and
an agent that calls send_message while also streaming prose posts exactly
once with the fallback silent.
Forward-ported onto main from the unmerged origin/micspiral/mesh-0-74-gemma
branch (bb4af41). This exact bug was hit in production by the "Local LLM
(llm1)" self-hosted mesh-llm/llama-server agent: turns ended with EndTurn
and correct prose visible in the ACP activity log, but nothing was published
to the channel because send_message was never called.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
with a helping hand from Claude Code
Signed-off-by: Brett Meehan <repudi8or@gmail.com>
Co-authored-by: imp0stor <imp0stor@users.noreply.github.com>
…ly delivery
The plain-reply-delivery fallback landed for the built-in relay-mesh
("Buzz shared compute") preset only, gated on provider == "relay-mesh"
in apply_relay_mesh_env. A self-hosted model reached the same way small
local models always are — the "OpenAI-compatible" provider pointed at a
user's own llama.cpp/vLLM/mesh-llm endpoint instead of the built-in
preset — never opted in, so the exact same silent-drop bug reproduced:
the model answers in prose, ACP's activity log shows a correct reply,
and nothing is ever published to the channel.
Default BUZZ_ACP_DELIVER_PLAIN_REPLIES to "true" for effective provider
"openai-compat" too, in the same effective-env assembly step, right
after the relay-mesh translation. Scoped narrowly to "openai-compat":
plain "openai" (the real OpenAI cloud API) is left untouched, since
cloud models reliably call send_message themselves and enabling the
fallback there would risk double-posting. An explicit user-set value
(including an intentional "false") is preserved, matching the emptyness
convention used elsewhere in this module.
Reproduced against the "Local LLM (llm1)" agent in production: provider
"openai-compat", OPENAI_COMPAT_BASE_URL pointed at a remote llama-server
instance — confirmed via the agent's own config screenshots that it is
not on the relay-mesh preset, so relay_mesh.rs's existing default never
applied to it.
with a helping hand from Claude Code
Signed-off-by: Brett Meehan <repudi8or@gmail.com>
Co-authored-by: imp0stor <imp0stor@users.noreply.github.com>
rmcp/schemars-derived tool schemas (buzz-dev-mcp's shell, read_file, str_replace, etc.) routinely carry JSON Schema metadata and assertions like $schema, title, format, and additionalProperties. Anthropic and the official OpenAI API tolerate these, but some openai-compat backends reached via a self-hosted endpoint or a multi-provider proxy (observed via a litellm-fronted OpenRouter route) hard-reject the whole tool-call request with "unsupported assertions or reserved metadata" instead of ignoring the extra keys. Add sanitize_tool_schema, applied once at tool registration in mcp.rs alongside the existing cap_schema size cap, to strip a denylist of pure-metadata and format/range-assertion keywords from every object in the schema tree. Every stripped keyword only narrows or documents what the schema accepts, never widens it, so removal can loosen validation but never break a previously-valid tool call. Structural/semantic keywords (type, properties, required, items, description, enum) are left untouched. Signed-off-by: repudi8or <repudi8or@gmail.com>
An earlier commit on this branch grew readiness.rs past its grandfathered 1742-line ratchet allowance (to 1850 lines), blocking the pre-push desktop-check hook. Following this repo's existing convention for oversized test modules (migration.rs's migration_tests.rs, migration_command_tests.rs, etc., and readiness.rs's own readiness_goose_file_config_tests.rs), extract the inline `mod tests` block into sibling files declared via #[path]. The extracted module still exceeded the 1000-line cap for new files, so it's split further into readiness_tests.rs and readiness_provider_tests.rs. readiness.rs itself drops to 690 lines. Also includes a cargo fmt fix to relay_mesh.rs (a pre-existing line-wrap picked up by running fmt across the whole crate), and reformats the two new test files to match. Signed-off-by: repudi8or <repudi8or@gmail.com> Co-authored-by: imp0stor <imp0stor@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR rebases and supersedes block#4874, enabling plaintext ACP reply delivery for self-hosted OpenAI-compatible endpoints and Ollama. Small local models frequently end a turn by streaming the answer as prose instead of calling
send_message, causing replies to be silently dropped. This adds an ACP harness fallback that publishes streamed text when no agent-authored channel reply was observed.Changes
ACP plaintext fallback (
buzz-acp): New--deliver-plain-repliesflag andBUZZ_ACP_DELIVER_PLAIN_REPLIESenv var. When enabled, the ACP harness captures assistant text during each turn and publishes it as a channel reply if the agent didn't callsend_message. Includes:Desktop readiness defaults: Self-hosted
openai-compatagents (llama.cpp, vLLM, local mesh-llm) and the built-inrelay-meshpreset now defaultBUZZ_ACP_DELIVER_PLAIN_REPLIEStotrue. Real OpenAI cloud API stays opt-in (cloud models reliably publish their own replies).MCP schema sanitization (
buzz-agent): OpenAI-compat backends reject MCP tool schemas withanyOf/allOf/$ref. The agent now sanitizes these before registration.Test refactor (
desktop): Splits oversizedreadiness.rstest module into sibling files to stay under the file-size ratchet.Related issues
Testing
cargo test -p buzz-acppassesOSS users running self-hosted models (llama.cpp, vLLM, Ollama, local mesh-llm) will benefit immediately: agents will no longer silently drop replies when they stream the answer as prose instead of calling
send_message.