Skip to content

fix(seahorse): prevent tool-call format leakage into LLM summaries#3279

Open
MrTreasure wants to merge 1 commit into
sipeed:mainfrom
MrTreasure:fix/seahorse-tool-call-leak
Open

fix(seahorse): prevent tool-call format leakage into LLM summaries#3279
MrTreasure wants to merge 1 commit into
sipeed:mainfrom
MrTreasure:fix/seahorse-tool-call-leak

Conversation

@MrTreasure

Copy link
Copy Markdown

Related: same symptom triggered by seahorse summaries

I just hit this exact class of bug — tool-call format leaking into user messages — but triggered through a different path: seahorse's partsToReadableContent.

The seahorse variant

partsToReadableContent (in pkg/seahorse/store.go) serializes tool calls as [tool_use: name, args: {...}] for FTS5 indexing. This format was also used as a fallback in two summary-generation paths (formatMessagesForSummary and truncateSummary in short_compaction.go). When summaries containing [tool_use: exec, args: {...}] were fed back into the LLM context, the model learned to mimic this format in its own responses — outputting mock tool calls as visible text content.

The pipeline

partsToReadableContent -> "[tool_use: exec, args: {...}]"
     -> seahorse summary / truncateSummary
     -> LLM context (as summary text)
     -> LLM mimics format in its content output
     -> User sees raw [tool_use: exec, args: {...}] in chat

Fix

PR incoming at: https://github.com/sipeed/picoclaw/pulls (MrTreasure:fix/seahorse-tool-call-leak)

Three changes:

  1. short_compaction.go: Skip messages with empty content in summary/truncation paths — tool calls aren't conversational content
  2. store.go: Change FTS5 format from [tool_use: name, args: {...}] to emoji markers — the bracket-colon pattern is what LLMs latch onto
  3. agent_utils.go: Strip residual [tool_use: ...] from outbound messages as a safety net

Same root cause as this issue: LLM context leaking internal serialization format → model mimics it → user sees raw tool-call text. Different trigger (seahorse vs Doubao provider), same outcome.

partsToReadableContent used [tool_use: name, args: {...}] format for
FTS5 indexing and summary generation. When summaries containing this
format were fed back to LLMs, the models learned to mimic it and output
mock [tool_use: ...] text as content, which was then displayed to users.

Two fixes:
1. formatMessagesForSummary / truncateSummary: skip messages with empty
   content (pure tool calls/results) — these are mechanical, not
   conversational, and have no place in summaries.
2. partsToReadableContent: use emoji markers (🔧/📋) instead of the
   bracket-colon format that LLMs are prone to mimic.
3. outboundMessageForTurn: strip any residual [tool_use: ...] patterns
   from content before sending to channels, as a safety net.

Root cause: same class of bug as sipeed#3153 (Doubao <seed:tool_call> leak).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant