[OpenAI] Migration to the Responses API?

[bilelomrani1]

I was wondering whether there are any plans to migrate instructor from the Chat Completions API to the new Responses API. OpenAI now recommends using the Responses API for new projects and highlights several potential benefits. Some of these may or may not be relevant to instructor, but here are three that caught my attention:

1. Lower costs

OpenAI claims:

Lower costs: Results in lower costs due to improved cache utilization (40% to 80% improvement when compared to Chat Completions in internal tests).

It’s difficult to know whether instructor would actually benefit from this without more details about how the Responses API differs internally. Still, this seems worth considering.

2. Stateful context

OpenAI also notes:

Stateful context: Use store: true to maintain state from turn to turn, preserving reasoning and tool context from turn to turn.

This could potentially be useful for instructor’s re-asking loop.

3. Future-proofing

Finally:

Future-proof: Future-proofed for upcoming models.

Depending on how this is interpreted, it might imply that newer models could eventually drop support for the older Chat Completions API.

[jxnl]

instructor/instructor/mode.py

Line 28 in 646075a

RESPONSES_TOOLS = "responses_tools"

Just use this mode, and everything will work.

[bilelomrani1]

Cool, thanks @jxnl I missed that one for some reason. Looks good, that solves the issue!

I see that the TOOLS mode is still the recommended mode for OpenAI in the docs (and the default used by .from_openai). The same goes for Mistral, the more recent MISTRAL_STRUCTURED_OUTPUTS mode could reasonably become the new default. What do you think about this?

[KennyVaneetvelde]

OpenAI's responses API optionally also returns the internal reasoning of the model - any way to access this?

[jxnl]

@cursoragent OpenAI's responses API optionally also returns the internal reasoning of the model - any way to access this?

[cursor]

At the moment, no. OpenAI does not return the full internal reasoning text in the Responses API. We only get the final output plus usage fields like reasoning token counts. Instructor does not expose a reasoning field either. If OpenAI adds a stable reasoning field, we can surface it. Until then, the best you can do is inspect the raw response via model._raw_response or create_with_completion, but it will not include chain-of-thought text.

Follow-up hot keys:

• [J] Add a short doc note that reasoning text is not exposed.
• [K] Add a small example showing _raw_response access.
• [L] Add a hook example to log full Responses API output.

 

[jxnl]

Closing as a roadmap/question item. If you'd like to track this, please open a Discussion (or a feature request with concrete scope + use cases).