-
Notifications
You must be signed in to change notification settings - Fork 108
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Description
Consider the following script:
from mirascope import llm
@llm.call("anthropic/claude-sonnet-4-5", format=int)
def lucky_number():
return "Choose a lucky number between 1 and 10"
result = lucky_number()
second_result = result.resume("Ok, now choose a different lucky number")This is very simple and should work. However, it raises the following error:
mirascope.llm.exceptions.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages.2: `tool_use` ids were found without `tool_result` blocks immediately after: toolu_01NUbw5mvfYRWaDHvMi1xYK1. Each `tool_use` block must have a corresponding `tool_result` block in the next message.'}, 'request_id': 'req_011CXxuFSqH1PCSqbYYrukMX'}
The issue is that:
- Anthropic provider uses tool mode by default
- Tool mode injects a special MIRASCOPE_FORMAT_TOOL, which is used for the output
- When Mirascope processes the assistant message, it converts the tool call into a text block that contains the expected output. There is never a corresponding user block with tool output (because the format tool is never called)
- When re-encoding the message history to send to Anthropic, we use the raw representation (unprocessed) which still has a tool call
- Anthropic rejects the request because the tool was never called
Solutions that come to mind include:
- When encoding the assistant message, check if it contained a format tool invocation, and if so re-generate it to have the text output, not the tool call. Sub-optimal because it may lose reasoning / thinking continuity and (if Anthropic auto-cached its output tokens) invalidate the cache. Also just feels a little hacky.
- Inject an additional user message invoking the tool when resuming the conversation. Also feels a little hacky — and in that case will Anthropic reject the subsequent real user message as violating turn order expectations?
Note, this issue is particularly pressing because:
- Tool use is the default structured format mode for Anthropic
- Response.validate() is a first class API that wraps a resume loop under the hood
However, we can mitigate the impact of this bug by switching Anthropic to use strict formatting by default (by opting into the beta) for all models that support it.
Thanks to Dahlia-claw at Reverie for finding and reporting this bug.
Python, Mirascope & OS Versions, related packages (not required)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working