fix(openai): normalize tool schemas before OpenAI / Responses / Codex requests

[IstvanBenedek]

Summary

OpenAI-backed Goose providers can reject valid Goose tool schemas before normal generation starts.

Observed failure:

Invalid schema for function 'autovisualiser__render_donut'

The root issue is not limited to one tool. Goose can emit JSON Schema constructs that OpenAI rejects during tool registration, including:

• $defs
• $ref
• anyOf / oneOf
• recursive local references
• nullable unions

This patch normalizes tool parameter schemas into an OpenAI-compatible subset before serializing requests for:

• OpenAI chat completions
• OpenAI responses
• ChatGPT Codex

Why This Approach

The narrower donut-only fix is useful, but it solves one manifestation of a broader provider-compatibility problem.

This patch keeps tool definitions and runtime behavior intact, and instead fixes the serialization boundary where OpenAI rejects the payload.

Implementation

In crates/goose/src/providers/formats/openai.rs this change:

• resolves local $ref entries from $defs / definitions
• flattens anyOf / oneOf
• merges allOf
• strips null-only variants from unions
• degrades recursive refs to a shallow generic object schema
• ensures object and array schemas still include the basic fields OpenAI expects

The same normalization is then applied when building requests in:

• crates/goose/src/providers/formats/openai.rs
• crates/goose/src/providers/formats/openai_responses.rs
• crates/goose/src/providers/chatgpt_codex.rs

Validation

Focused regression tests:

• cargo test -p goose --no-default-features --features rustls-tls test_validate_tool_schemas -- --nocapture
• cargo test -p goose --no-default-features --features rustls-tls test_create_request_sanitizes_tool_schema -- --nocapture
• cargo test -p goose --no-default-features --features rustls-tls test_codex_request_sanitizes_tool_schema -- --nocapture
• cargo test -p goose --no-default-features --features rustls-tls test_responses_request_sanitizes_tool_schema -- --nocapture

Manual validation:

• source-built CLI no longer fails on the original schema error with autovisualiser enabled
• interactive CLI path succeeds
• packaged desktop app launches with the same patched bundled binaries

Reviewer Notes

This is intentionally a compatibility transform, not an exact schema-preserving rewrite.

The tradeoff is:

• outgoing schemas may become somewhat broader
• Goose no longer fails at request time on OpenAI-backed providers that reject advanced schema constructs

The patch is limited to provider-facing request shaping and does not change tool execution behavior.

Related Issue

• Fixes autovisualiser__render_donut schema rejected by OpenAI-compatible tool calling #8173

I’ll add a follow-up PR comment with a concise reviewer guide and diagrams showing the request-shaping flow and a concrete before/after schema example.

[IstvanBenedek]

Supplemental review context for this PR.

These notes mirror the local review pack I used while validating the change. The two Mermaid diagrams below are the same explanation that I exported locally as PNGs; I’m keeping the binary assets out of the repo and putting the reviewer context here instead.

What changed

The fix is intentionally provider-side.

It does not change tool runtime behavior or autovisualiser rendering logic.
It only rewrites tool parameter schemas at the OpenAI-facing request boundary so Goose stops sending JSON Schema constructs that OpenAI rejects.

Affected request paths:

• OpenAI chat completions
• OpenAI responses
• ChatGPT Codex

What to look at first

The main logic is in crates/goose/src/providers/formats/openai.rs.

The key reviewer question is not whether the outgoing schema is a byte-for-byte rewrite of the original. It is whether the outgoing schema stays valid enough for tool use while avoiding shapes that OpenAI rejects.

Main tradeoff:

• some outgoing schemas may become broader than the original source schema
• Goose no longer fails before generation on OpenAI-backed providers that reject $defs, $ref, anyOf, recursive refs, and similar shapes

Diagram: request-shaping flow

flowchart TD
    A["Tool input_schema from rmcp/schemars"] --> B["OpenAI-facing request builder"]
    B --> C{"Schema contains $defs / $ref / anyOf / recursive refs?"}
    C -->|No| D["Keep schema unchanged"]
    C -->|Yes| E["Normalize schema"]
    E --> F["Resolve local refs"]
    E --> G["Flatten unions"]
    E --> H["Break recursive refs"]
    E --> I["Ensure object/array completeness"]
    D --> J["OpenAI-safe tool schema"]
    F --> J
    G --> J
    H --> J
    I --> J
    J --> K["Serialize request to OpenAI / Responses / Codex"]

Loading

Diagram: concrete before/after example

flowchart LR
    BEFORE["Before<br/><br/>$defs: SingleDonutChart<br/>data.anyOf: $ref | array($ref)<br/>values.items.anyOf: number | object(label, value)"] --> X["normalize_json_schema_for_openai()"]
    X --> AFTER["After<br/><br/>No $defs / $ref / anyOf<br/>data.type: [object, array]<br/>data.items.type: object<br/>values.items.type: [number, object]"]
    AFTER --- NOTE["Same tool intent<br/>smaller schema surface for OpenAI"]

Loading

Validation

Focused regression tests run locally:

• cargo test -p goose --no-default-features --features rustls-tls test_validate_tool_schemas -- --nocapture
• cargo test -p goose --no-default-features --features rustls-tls test_create_request_sanitizes_tool_schema -- --nocapture
• cargo test -p goose --no-default-features --features rustls-tls test_codex_request_sanitizes_tool_schema -- --nocapture
• cargo test -p goose --no-default-features --features rustls-tls test_responses_request_sanitizes_tool_schema -- --nocapture

Practical validation also passed locally with a patched install and autovisualiser enabled.