-
Notifications
You must be signed in to change notification settings - Fork 3.6k
fix(openai): handle image-url tool result content type in responses API #11488
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(openai): handle image-url tool result content type in responses API #11488
Conversation
|
hi thanks for the PR!
yes please - add it in the examples folder in the appropriate sub directory so that we can verify it too |
|
|
Sure thing, will pull something together and update. |
|
Example added for testing 👍 |
aayush-kapoor
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
verified that the fix works for the example
Background
The AI SDK defines
image-urlas a valid tool result content type inLanguageModelV3ToolResultOutput, but the OpenAI provider'sconvertToOpenAIResponsesInputfunction didn't handle this type.When a tool returned
image-urlcontent, it fell through to the default case, produced a warning ("unsupported tool content part type: image-url"), and the image was dropped.Summary
Added a case to handle
image-urltool result content by converting it to OpenAI'sinput_imageformat. OpenAI's Responses API accepts HTTP URLs in theimage_urlfield (alongside data URLs), so this is a gap in the provider implementation rather than a missing API feature.Manual Verification
I tested these changes in a separate project with a tool that returns
image-urlcontent:I can provide a self-contained code sample if needed.
Checklist
pnpm changesetin the project root)