Skip to content

Add response_format support to model settings - #258

Open
juliendenize wants to merge 2 commits into
mainfrom
feat/response-format-model-settings
Open

Add response_format support to model settings#258
juliendenize wants to merge 2 commits into
mainfrom
feat/response-format-model-settings

Conversation

@juliendenize

Copy link
Copy Markdown
Collaborator

Why

Some models constrain their output to a JSON schema. To support this end-to-end, the encoded ModelSettings (which is what the tokenizer emits to condition the model) needs to carry the JSON schema derived from a request's response_format. This adds first-class response_format / JSON-schema support so those models can be driven correctly through mistral-common.

Summary

  • Adds json_schema support to ResponseFormat:
    • New ResponseFormats.json_schema enum value.
    • New JsonSchema model (name, description, schema aliased to custom_schema, strict) with Draft 7 validation.
    • ResponseFormat.get_schema() resolves the effective schema: custom schema for json_schema, generic object/array anyOf for json_object, None for text.
  • Adds json_schema field to encoded ModelSettings (tokenizer >= v15).
  • Reworks ModelSettingsBuilder / FieldBuilder to a typed InputT -> OutputT convert-then-validate design so a request field (e.g. response_format) can be converted into a different settings field (e.g. json_schema), via _SETTINGS_TO_CONV_FIELDS_MAP.
  • New JSONSchemaBuilder converts a ResponseFormat to a validated schema dict.
  • New mistral_common/utils/json_utils.py with validate_json_schema_by_draft7.
  • ChatCompletionRequest.to_openai now serializes by_alias=True so schema is emitted correctly.

Reviewer guide

Real logic:

  • src/mistral_common/protocol/instruct/request.py — new JsonSchema, ResponseFormat.get_schema(), ModelSettings.json_schema, by_alias serialization.
  • src/mistral_common/tokens/tokenizers/model_settings_builder.py — the FieldBuilder generic refactor (convert vs validate split) and new JSONSchemaBuilder + field mapping. Most careful review here.

New:

  • src/mistral_common/utils/json_utils.py (+ utils/__init__.py) — Draft 7 schema validation helper.

Tests:

  • tests/test_model_settings.py, tests/test_request.py, tests/test_json_utils.py, tests/test_converters.py, tests/test_tokenizer_v15.py — coverage for the new behavior.

Test plan

  • pytest tests/test_model_settings.py tests/test_request.py tests/test_json_utils.py tests/test_converters.py tests/test_tokenizer_v15.py — 171 passed.
  • ruff format --check, ruff check, mypy on changed sources — all clean.

@juliendenize
juliendenize force-pushed the feat/response-format-model-settings branch from 6043d64 to 7b675a3 Compare July 7, 2026 12:53
@juliendenize
juliendenize force-pushed the feat/response-format-model-settings branch from 7b675a3 to bc92712 Compare July 7, 2026 13:02
@juliendenize
juliendenize marked this pull request as ready for review July 7, 2026 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant