-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add tool approval integration for Vercel AI adapter #3772
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
099f07a to
1160591
Compare
|
Thanks for the quick reviews! Will get the test issue fixed and reply to your comments shortly. |
2deeb25 to
9eebd80
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll take a closer look at docs later, will focus primarily on the code for now
|
This PR is stale, and will be closed in 3 days if no reply is received. |
|
Closing this PR as it has been inactive for 10 days. |
|
Have been on a break for a bit, will get the comments addressed early this week if not today. |
|
@bendrucker Thanks Ben, I should probably have disabled the stale bot before the holidays :) |
|
Thanks for your patience! Made the requested changes and provided a reference to relevant AI SDK test snapshots for the behavioral question. |
|
Squashing another upstream cause of a test flake: pytest-dev/pytest-xdist#1299 |
|
@bendrucker I merged some other changes for the Vercel AI event stream that had been in the works for a few weeks -- can you resolve the conflicts please? |
|
Resolved! |
| return cls( | ||
| agent=agent, | ||
| run_input=cls.build_run_input(await request.body()), | ||
| accept=request.headers.get('accept'), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldn't be repeating super class implementation here; can we call the super method and then set enable_tool_approval directly on the returned instance?
We could also choose not to have this method override at all, and just tell the user to set the flag like that directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that if we override methods, we should also support this flag on dispatch_request, as that's the main way people use these UI adapters. If that's hard to do, perhaps it'd be better to always have the user build the adapter, set the flag, and then call streaming_response as in your example.
I also wonder if there's a way we can detect from the incoming API request whether the frontend is using AI SDK UI v6 or later? In that case we may not need the flag at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also wonder if there's a way we can detect from the incoming API request whether the frontend is using AI SDK UI v6 or later? In that case we may not need the flag at all.
Good call, I'll look into whether they have a protocol version or whether they do feature detection based on the requests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The extent of explicit versioning is the x-vercel-ai-ui-message-stream: v1 header. It comes from the server. There's no Accept header or similar negotiation from client to server.
The client is strict about what the server sends, only known chunk types, only known keys:
Any validation error against this schema interrupts streaming.
There's nothing we can detect in the client requests on the initial interaction. We'd send a tool approval request to the client and if it's v5 it just would never be able to provide an approval-responded part.
|
@bendrucker Please have a look at the conflicts |
- Inline extract_deferred_tool_results logic into from_request() - Remove unit tests for private _extract_deferred_tool_results method - Remove unit tests for private _denied_tool_ids property - Update test_tool_output_denied_chunk_emission to use public interface
- Update test_tool_output_denied_chunk_emission to use from_request() with explicit type binding to test the full public interface - Remove test_from_request_with_tool_approval_enabled (now redundant) - Remove test_deferred_tool_results_fallback_from_instance (tested internal plumbing rather than observable behavior)
- Rename tool_approval to enable_tool_approval (add enable_ prefix) - Make deferred_tool_results a cached_property instead of instance field - Use explicit loops for type narrowing in approval extraction - Simplify tests: remove mocking, use snapshots, move imports to top
- Add run_id=IsStr() to message snapshot (messages do have run_id) - Apply ruff import sorting fixes
e8e8ccc to
ac27619
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
@bendrucker A few of my earlier comments remain unresolved, right? |
|
Hmm I thought I remember addressing those but I will take a look later and see what happened to that work. |
|
Resolved:
|
f325008 to
975a215
Compare
|
@bendrucker Once #4166 is merged (should be shortly), please update to use the sdk version flag to auto-enable this instead of the new boolean. You can mention in the docstring for the field that it enables approval handling. |
Replace enable_tool_approval boolean with sdk_version parameter: - sdk_version=6 enables tool approval streaming (human-in-the-loop) - sdk_version=5 (default) disables tool approval for backward compat Updated adapter, event stream, tests, and docs accordingly.
8637777 to
c19330a
Compare
|
Merged! Totally forgot that somewhere in this PR I investigated the frontend side and found strict schema handling via zod, with errors on unknown keys. Not sure that's sensible/necessary behavior on AI SDK's part, but given that reality, a version attribute is definitely the right move for Pydantic AI. Certainly simplifies things here. |
Adds tool approval integration for the Vercel AI adapter, enabling human-in-the-loop workflows with AI SDK UI.
Summary
enable_tool_approvalflag for simplified tool approval workflowstool-approval-requestchunks for deferred tool approvalstool-output-deniedchunks when user denies tool executionapprovalfield to tool parts withToolApprovalRequested/ToolApprovalRespondedstatesUsage
When
enable_tool_approval=True, the adapter will:tool-approval-requestchunks when tools withrequires_approval=Trueare calledtool-output-deniedchunks for rejected tools, passing the denialreasonthrough asToolDeniedAI SDK Tool Approval Protocol
Tool approval is an AI SDK v6 feature. Here's how the protocol works:
Protocol Flow
sequenceDiagram participant Model participant Server participant Client Model->>Server: tool call Server->>Client: tool-input-start Server->>Client: tool-input-available Server->>Client: tool-approval-request Note over Client: User approves/denies Client->>Server: approval response (next request) alt Approved Server->>Server: Execute tool Server->>Client: tool-output-available else Denied Server->>Client: tool-output-denied Server->>Model: denial info endChunk Types
tool-approval-requestServer → client:
{ "type": "tool-approval-request", "approvalId": "<uuid>", "toolCallId": "<tool-call-id>" }tool-output-deniedServer → client, when denied:
{ "type": "tool-output-denied", "toolCallId": "<tool-call-id>" }Why the Server Emits This Chunk
This is correct—the denial decision originates from the user via the
ToolApprovalRespondedfield in tool parts. However, the AI SDK protocol expects the server to emittool-output-deniedfor two reasons:The client needs confirmation that the tool lifecycle is complete. When the server receives a denial and emits
tool-output-denied, the client can transition its UI from "awaiting result" to "denied".Just as the server emits
tool-output-availablewhen a tool executes successfully, it emitstool-output-deniedwhen execution is skipped due to denial. This gives the client a consistent signal for each tool call's final state.The flow is:
approval: { id, approved: false, reason }in the tool partdeferred_tool_resultsand passes it to the agenttool-output-deniedto the clientTool Part Approval States
Tool parts include an
approvalfield tracking the approval lifecycle:Awaiting Response
{ "id": "<approval-id>" }User Responded
{ "id": "<approval-id>", "approved": true/false, "reason": "optional" }Two-Step Flow
Unlike regular tool calls, approved tools require two model interactions:
tool-approval-request→ awaits userChanges
ToolApprovalRequested/ToolApprovalRespondedtypes andapprovalfield to tool UI partsToolApprovalRequestChunkandToolOutputDeniedChunkresponse chunks to event streamenable_tool_approvalparameter toVercelAIAdapter.from_request()with auto-extraction of approval responses viadeferred_tool_resultscached propertyreasonthrough asToolDeniedwhen providedTesting
approvalfield on tool partsReferences