fix: consistent error handling between run_sync and run_stream for invalid output tool calls#4860
fix: consistent error handling between run_sync and run_stream for invalid output tool calls#4860Spectual wants to merge 3 commits intopydantic:mainfrom
Conversation
…valid output tool calls Fixes pydantic#3638 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
github-actions
bot
added
size: S
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| except (ValidationError, exceptions.ToolRetryError) as e: | ||
| if not allow_partial: | ||
| raise exceptions.UnexpectedModelBehavior('Invalid output') from e | ||
| raise |
There was a problem hiding this comment.
🚩 ModelRetry not caught by the new except block
The new except at pydantic_ai_slim/pydantic_ai/result.py:245 catches ValidationError and ToolRetryError but not ModelRetry. ModelRetry can propagate from handle_call (via validate_tool_call with wrap_validation_errors=False at pydantic_ai_slim/pydantic_ai/_tool_manager.py:393-394) when a tool's validation hooks raise ModelRetry, or from the text processor path (via execute_traced_output_function at pydantic_ai_slim/pydantic_ai/_output.py:144-145). If ModelRetry is raised during final validation (allow_partial=False), it would leak to the user unwrapped — the same class of issue this PR fixes for ValidationError. However, this is pre-existing behavior unchanged by this PR, so it's an incomplete fix rather than a regression. In practice, ModelRetry from output tool validation is uncommon in the streaming context.
Was this helpful? React with 👍 or 👎 to provide feedback.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Spectual
force-pushed
the
fix/streaming-output-validation
branch
from
4400d63 to
37edf30
Compare
…e_output Merge the two identical `except (ValidationError, ToolRetryError)` blocks into a single try/except wrapping the entire if/elif chain, per AGENTS.md rule 894. The intermediate branches (deferred_tool_requests, allows_image) do not raise these exceptions so the consolidation is safe.
github-actions
bot
added
size: M
Spectual
force-pushed
the
fix/streaming-output-validation
branch
from
2cef5c1 to
98bba49
Compare
1 participant
Summary
Fix inconsistent behavior between
agent.run_syncandagent.run_streamwhen no valid output tool calls are returned by the model.Problem
As reported in #3638, when the model returns no valid output tool calls:
agent.run_syncraisesUnexpectedModelBehavioragent.run_stream+get_output()raisespydantic_core.ValidationErrorThis inconsistency makes it difficult to write reliable error handling code.
Solution
Updated the streaming output validation in
result.pyto catchValidationErrorduring output processing and convert it toUnexpectedModelBehavior, matching the behavior of the non-streaming path.Testing
Fixes #3638