Skip to content

test: do tool calls improve with openai input schema style?#692

Draft
phryneas wants to merge 3 commits intomainfrom
pr/test-openai-input-schema-style
Draft

test: do tool calls improve with openai input schema style?#692
phryneas wants to merge 3 commits intomainfrom
pr/test-openai-input-schema-style

Conversation

@phryneas
Copy link
Copy Markdown
Member

@phryneas phryneas commented Mar 11, 2026

@jerelmiller has told me that a number of tools just don't get called right by some agents, so my suspicion is that these tools first try to "just generate the tool call", and if that doesn't work apply a step similar to the Vercel SDK's experimental_repairToolCall, in which they probably try to call the model's variant of generateObject with the inputSchema.
In the case of OpenAI models, the JSON Schema specification for that kind of call is very restricted, see their documentation.

Most importantly: there are no optional fields - every field has to be required, and additionalProperties always needs to be false.

Our schemas currently don't fit these requirements - so let's just try if that improves the situation.

Obviously, this is not a fully functional PR, and tons of tests are still failing, but it should be enough for Jerel to build this locally and try out if the quality of tool calls improves.

Note: there are a lot of additional subtleties, so please inspect the generated schema if it really matches the OpenAI requirements before drawing any conclusions from this 😅

@apollo-librarian
Copy link
Copy Markdown
Contributor

apollo-librarian bot commented Mar 11, 2026

✅ Docs preview has no changes

The preview was not built because there were no changes.

Build ID: 767af3ec709be0e123d37128
Build Logs: View logs


✅ AI Style Review — No Changes Detected

No MDX files were changed in this pull request.

Review Log: View detailed log

This review is AI-generated. Please use common sense when accepting these suggestions, as they may not always be accurate or appropriate for your specific context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant