Commit 79e913d
authored
fix(openai): remove duplicate schema from messages in JSON_SCHEMA mode (#1761)
Removes redundant schema information from messages when using
`JSON_SCHEMA` mode.
### Why This Change?
**JSON mode** (`response_format: {"type": "json_object"}`) - OpenAI docs
require explicit JSON instruction in messages since no schema is
provided in response_format.
https://platform.openai.com/docs/guides/structured-outputs?api-mode=chat#json-mode
> When using JSON mode, you must always instruct the model to produce
JSON via some message in the conversation, for example via your system
message. If you don't include an explicit instruction to generate JSON,
the model may generate an unending stream of whitespace and the request
may run continually until it reaches the token limit. To help ensure you
don't forget, the API will throw an error if the string "JSON" does not
appear somewhere in the context.
**JSON_SCHEMA mode** (`response_format: {"type": "json_schema", ...}`) -
Schema is already provided in response_format. Adding the same schema to
messages creates redundancy, increases token consumption unnecessarily,
and provides no additional value to the model.
### Changes
- `JSON_SCHEMA` mode: No schema added to messages (schema already in
response_format)
- `JSON` and `MD_JSON` modes: Unchanged behavior (still add schema to
messages as required)
<!-- ELLIPSIS_HIDDEN -->
----
> [!IMPORTANT]
> Removes redundant schema from messages in `JSON_SCHEMA` mode in
`handle_json_modes()` in `utils.py`, reducing token consumption.
>
> - **Behavior**:
> - In `handle_json_modes()` in `utils.py`, `JSON_SCHEMA` mode no longer
adds schema to messages, as it's already in `response_format`.
> - `JSON` and `MD_JSON` modes remain unchanged, still adding schema to
messages.
> - **Rationale**:
> - Reduces redundancy and token consumption in `JSON_SCHEMA` mode by
not duplicating schema in messages.
>
> <sup>This description was created by </sup>[<img alt="Ellipsis"
src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=567-labs%2Finstructor&utm_source=github&utm_medium=referral)<sup>
for f3af7fb. You can
[customize](https://app.ellipsis.dev/567-labs/settings/summaries) this
summary. It will automatically update as commits are pushed.</sup>
<!-- ELLIPSIS_HIDDEN -->1 file changed
+17
-16
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
439 | 439 | | |
440 | 440 | | |
441 | 441 | | |
442 | | - | |
443 | | - | |
444 | | - | |
445 | | - | |
446 | | - | |
447 | | - | |
448 | | - | |
449 | | - | |
450 | | - | |
451 | | - | |
452 | | - | |
453 | | - | |
454 | | - | |
455 | | - | |
456 | | - | |
457 | | - | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
458 | 459 | | |
459 | 460 | | |
460 | 461 | | |
| |||
0 commit comments