Skip to content

Commit 79e913d

Browse files
authored
fix(openai): remove duplicate schema from messages in JSON_SCHEMA mode (#1761)
Removes redundant schema information from messages when using `JSON_SCHEMA` mode. ### Why This Change? **JSON mode** (`response_format: {"type": "json_object"}`) - OpenAI docs require explicit JSON instruction in messages since no schema is provided in response_format. https://platform.openai.com/docs/guides/structured-outputs?api-mode=chat#json-mode > When using JSON mode, you must always instruct the model to produce JSON via some message in the conversation, for example via your system message. If you don't include an explicit instruction to generate JSON, the model may generate an unending stream of whitespace and the request may run continually until it reaches the token limit. To help ensure you don't forget, the API will throw an error if the string "JSON" does not appear somewhere in the context. **JSON_SCHEMA mode** (`response_format: {"type": "json_schema", ...}`) - Schema is already provided in response_format. Adding the same schema to messages creates redundancy, increases token consumption unnecessarily, and provides no additional value to the model. ### Changes - `JSON_SCHEMA` mode: No schema added to messages (schema already in response_format) - `JSON` and `MD_JSON` modes: Unchanged behavior (still add schema to messages as required) <!-- ELLIPSIS_HIDDEN --> ---- > [!IMPORTANT] > Removes redundant schema from messages in `JSON_SCHEMA` mode in `handle_json_modes()` in `utils.py`, reducing token consumption. > > - **Behavior**: > - In `handle_json_modes()` in `utils.py`, `JSON_SCHEMA` mode no longer adds schema to messages, as it's already in `response_format`. > - `JSON` and `MD_JSON` modes remain unchanged, still adding schema to messages. > - **Rationale**: > - Reduces redundancy and token consumption in `JSON_SCHEMA` mode by not duplicating schema in messages. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=567-labs%2Finstructor&utm_source=github&utm_medium=referral)<sup> for f3af7fb. You can [customize](https://app.ellipsis.dev/567-labs/settings/summaries) this summary. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN -->
2 parents a9b25e7 + 90dd5b7 commit 79e913d

File tree

1 file changed

+17
-16
lines changed

1 file changed

+17
-16
lines changed

instructor/providers/openai/utils.py

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -439,22 +439,23 @@ def handle_json_modes(
439439
)
440440
new_kwargs["messages"] = merge_consecutive_messages(new_kwargs["messages"])
441441

442-
if new_kwargs["messages"][0]["role"] != "system":
443-
new_kwargs["messages"].insert(
444-
0,
445-
{
446-
"role": "system",
447-
"content": message,
448-
},
449-
)
450-
elif isinstance(new_kwargs["messages"][0]["content"], str):
451-
new_kwargs["messages"][0]["content"] += f"\n\n{message}"
452-
elif isinstance(new_kwargs["messages"][0]["content"], list):
453-
new_kwargs["messages"][0]["content"][0]["text"] += f"\n\n{message}"
454-
else:
455-
raise ValueError(
456-
"Invalid message format, must be a string or a list of messages"
457-
)
442+
if mode != Mode.JSON_SCHEMA:
443+
if new_kwargs["messages"][0]["role"] != "system":
444+
new_kwargs["messages"].insert(
445+
0,
446+
{
447+
"role": "system",
448+
"content": message,
449+
},
450+
)
451+
elif isinstance(new_kwargs["messages"][0]["content"], str):
452+
new_kwargs["messages"][0]["content"] += f"\n\n{message}"
453+
elif isinstance(new_kwargs["messages"][0]["content"], list):
454+
new_kwargs["messages"][0]["content"][0]["text"] += f"\n\n{message}"
455+
else:
456+
raise ValueError(
457+
"Invalid message format, must be a string or a list of messages"
458+
)
458459

459460
return response_model, new_kwargs
460461

0 commit comments

Comments
 (0)