Handle Anthropic `stop_reason=pause_turn` and OpenAI background mode by mattbrandman · Pull Request #4306 · pydantic/pydantic-ai

mattbrandman · 2026-02-12T03:02:45Z

Potentially replaces Add model continuation support (Anthropic pause_turn + OpenAI background mode) #4195
Closes Anthropic stop_reason pause_turn is not handled correctly, resulting in errors with long-running built-in tools #2600
Closes Support for OpenAI Background Mode #3268

mattbrandman · 2026-02-12T22:38:34Z

@DouweM this is a slightly different take leveraging the fact that we do in fact have a graph to work with.

pydantic_ai_slim/pydantic_ai/_agent_graph.py

pydantic_ai_slim/pydantic_ai/messages.py

pydantic_ai_slim/pydantic_ai/_agent_graph.py

pydantic_ai_slim/pydantic_ai/models/openai.py

pydantic_ai_slim/pydantic_ai/_agent_graph.py

mattbrandman · 2026-03-25T18:40:58Z

Status update: branch restructured and review feedback addressed

This PR has been restructured into a focused stack:

main
 → continuation-node-approach (this PR) — ContinueRequestNode, ModelResponseState, fallback pinning
   → skill-support-v2 (#4622)           — ShellTool, background mode, native tools, model adapters
     → local-tools                       — toolset tests, cassettes, docs

CI fixes in this push

Fixed ExceptionGroup NameError on Python 3.10 — added conditional import from exceptiongroup backport (matching existing pattern in tests/evals/test_utils.py)
Fixed inline snapshots for new state/metadata fields on ModelResponse serialization
Fixed ruff formatting

Review feedback addressed (from earlier commits + this restructure)

✅ Renamed ContinuationNode → ContinueRequestNode (per @DouweM)
✅ Removed empty ModelRequest from continuation — passes message_history directly
✅ Removed deprecated BuiltinToolCallEvent/BuiltinToolResultEvent emission
✅ Removed interrupted/incomplete from ModelResponseState — only complete/suspended (per @DouweM)
✅ Moved fallback routing state to ModelResponse.metadata with __pydantic_ai__ key (per @DouweM)
✅ ContinueRequestNode exported in __all__ (per @DouweM)

Remaining unresolved threads (will address in follow-up or are no longer applicable)

Most unresolved review threads from github-actions bot and devin-ai-integration are on code that now lives in skill-support-v2 (OpenAI background mode, shell tool container handling) and will be addressed there. The continuation-specific threads are resolved by the changes above.

devin-ai-integration

Devin Review found 1 new potential issue.

View 25 additional findings in Devin Review.

devin-ai-integration · 2026-03-25T19:24:35Z

pydantic_ai_slim/pydantic_ai/models/anthropic.py

+        # On pause_turn continuation, pass just the container ID string to reconnect.
+        # Re-passing BetaContainerParams triggers a prefill rejection on some models
+        # (e.g. Sonnet 4-6) even though plain string ID works fine.
+        if messages and isinstance(messages[-1], ModelResponse) and messages[-1].state == 'suspended':
+            if messages[-1].provider_details:
+                return messages[-1].provider_details.get('container_id')
+            return None


🚩 Anthropic _get_container returns raw string instead of BetaContainerParams for continuations

At anthropic.py:534-536, when the last message is a suspended response, _get_container returns the raw container_id string from provider_details instead of wrapping it in BetaContainerParams(id=...) as is done in the normal path at line 542. The comment explains this is intentional to avoid a prefill rejection on some models. This is a provider-specific workaround that may break if Anthropic changes their API contract. The return type annotation says BetaContainerParams | None but now it can also return str, which is a type inconsistency (though the API apparently accepts both).

Was this helpful? React with 👍 or 👎 to provide feedback.

devin-ai-integration

Devin Review found 1 new potential issue.

View 27 additional findings in Devin Review.

devin-ai-integration · 2026-03-26T17:46:22Z

pydantic_ai_slim/pydantic_ai/_agent_graph.py

+    @staticmethod
+    def _merge_response(existing: _messages.ModelResponse, new: _messages.ModelResponse) -> _messages.ModelResponse:
+        """Merge a new response into an existing one.
+
+        If same `provider_response_id`, replace entirely with the new response.
+        If the model changed between responses, replace entirely (incompatible responses should not be merged).
+        Otherwise, accumulate parts, sum usage, and use other fields from the new response.
+        """
+        # Same response ID → the new response is a full replacement (e.g. OpenAI background retrieve).
+        if existing.provider_response_id and existing.provider_response_id == new.provider_response_id:
+            return new
+
+        # Different model → replace (accumulating parts from different models is always wrong).
+        # When either model_name is None/empty, we fall through to accumulation — this is intentional
+        # because providers may not always populate model_name on continuation responses.
+        if existing.model_name and new.model_name and existing.model_name != new.model_name:
+            return new
+
+        # Same model, different response → accumulate parts and sum usage.
+        # Preserve existing provider response IDs when continuation responses omit them
+        # (e.g. resumed OpenAI streams that start after a sequence number).
+        merged_usage = existing.usage + new.usage
+        return replace(
+            new,
+            parts=[*existing.parts, *new.parts],
+            usage=merged_usage,
+            provider_response_id=new.provider_response_id or existing.provider_response_id,
+        )


🚩 _merge_response accumulates parts across continuations — duplicate parts possible if provider echoes earlier content

The merge logic at _agent_graph.py:1346-1351 concatenates [*existing.parts, *new.parts]. This works correctly for providers like Anthropic that return only NEW content in continuation responses. However, if a provider echoes earlier parts in the continuation response (e.g. some OpenAI retrieve responses return the full output), the merged response would contain duplicate parts. The same-provider_response_id check at line 1333 handles the OpenAI background case (where retrieve returns the full response with the same ID), but an edge case could arise if a provider returns a different response ID but includes earlier content.

Was this helpful? React with 👍 or 👎 to provide feedback.

…ck continuation pinning Adds general-purpose continuation infrastructure for models that pause mid-turn: - ContinueRequestNode in agent graph for automatic continuation requests - ModelResponseState type (complete/suspended) on ModelResponse - Fallback model continuation pinning: pin to the model that started a continuation - Message rewinding for fallback recovery This enables Anthropic pause_turn and OpenAI background mode (added in follow-up).

github-actions bot added the size: L Large PR (501-1500 weighted lines) label Feb 12, 2026

mattbrandman mentioned this pull request Feb 12, 2026

Add model continuation support (Anthropic pause_turn + OpenAI background mode) #4195

Closed

6 tasks

github-actions bot added the feature New feature request, or PR implementing a feature (enhancement) label Feb 12, 2026

DouweM reviewed Feb 16, 2026

View reviewed changes

pydantic_ai_slim/pydantic_ai/_agent_graph.py Outdated Show resolved Hide resolved

pydantic_ai_slim/pydantic_ai/_agent_graph.py Outdated Show resolved Hide resolved

pydantic_ai_slim/pydantic_ai/_agent_graph.py Outdated Show resolved Hide resolved

DouweM added the auto-review label Feb 16, 2026

This comment was marked as resolved.

Sign in to view

github-actions bot removed the auto-review label Feb 16, 2026

mattbrandman marked this pull request as ready for review February 18, 2026 14:20

This comment was marked as resolved.

Sign in to view

mattbrandman force-pushed the continuation-node-approach branch from 400bb22 to 2387b04 Compare February 19, 2026 18:59

mattbrandman mentioned this pull request Feb 19, 2026

Support files uploaded to providers via new UploadedFile object #3942

Merged

6 tasks

DouweM added the auto-review label Feb 19, 2026

This comment was marked as resolved.

Sign in to view

DouweM changed the title ~~Continuation node approach~~ Handle Anthropic stop_reason=pause_turn and OpenAI background mode Mar 3, 2026

DouweM requested changes Mar 3, 2026

View reviewed changes

DouweM reviewed Mar 3, 2026

View reviewed changes

pydantic_ai_slim/pydantic_ai/_agent_graph.py Outdated Show resolved Hide resolved

mattbrandman force-pushed the continuation-node-approach branch 2 times, most recently from bc560c5 to b757f1d Compare March 25, 2026 18:04

github-actions bot added size: L Large PR (501-1500 weighted lines) and removed size: XL Extra large PR (>1500 weighted lines) labels Mar 25, 2026

mattbrandman force-pushed the continuation-node-approach branch from b757f1d to 4e9533e Compare March 25, 2026 18:13

This comment was marked as resolved.

Sign in to view

mattbrandman force-pushed the continuation-node-approach branch from 4e9533e to 51c8b30 Compare March 25, 2026 18:40

mattbrandman force-pushed the continuation-node-approach branch from 51c8b30 to 7252260 Compare March 25, 2026 19:17

github-actions bot added size: XL Extra large PR (>1500 weighted lines) and removed size: L Large PR (501-1500 weighted lines) labels Mar 25, 2026

mattbrandman force-pushed the continuation-node-approach branch from 7252260 to 45ea637 Compare March 25, 2026 19:19

devin-ai-integration bot reviewed Mar 25, 2026

View reviewed changes

mattbrandman force-pushed the continuation-node-approach branch 8 times, most recently from b772648 to fb9442d Compare March 26, 2026 17:15

This comment was marked as resolved.

Sign in to view

mattbrandman force-pushed the continuation-node-approach branch from fb9442d to 0ec1274 Compare March 26, 2026 17:40

devin-ai-integration bot reviewed Mar 26, 2026

View reviewed changes

mattbrandman force-pushed the continuation-node-approach branch from 0ec1274 to 14446ac Compare March 26, 2026 18:07

Conversation

mattbrandman commented Feb 12, 2026 • edited by DouweM Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattbrandman commented Feb 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

mattbrandman commented Mar 25, 2026

Status update: branch restructured and review feedback addressed

CI fixes in this push

Review feedback addressed (from earlier commits + this restructure)

Remaining unresolved threads (will address in follow-up or are no longer applicable)

Uh oh!

devin-ai-integration bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration bot Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration bot Mar 26, 2026

Choose a reason for hiding this comment

mattbrandman commented Feb 12, 2026 •

edited by DouweM

Loading