Skip to content

Conversation

@Athroniaeth
Copy link

I use pydantic models and the examples attribute of the Field descriptor in my structured output projects whenever possible. I noticed in the Mistral documentation that there was a function for obtaining a schema, and I wanted to test it to compare the one I had made with yours, and I noticed that I had a strange error.

It seems that the int was omitted from what I assume to be the list of primary types in the rec_strict_json_schema function.

Here's how to reproduce the error:

from mistralai.extra import response_format_from_pydantic_model
from pydantic import BaseModel, Field


class Pass1(BaseModel):
    string: int = Field(...)
    boolean: bool = Field(...)
    number: int = Field(...)

class Pass2(BaseModel):
    string: int = Field(..., examples=["example1", "example2"])
    boolean: bool = Field(..., examples=[True, False])
    number: int = Field(...)


class Failed(BaseModel):
    string: int = Field(..., examples=["example1", "example2"])
    boolean: bool = Field(..., examples=[True, False])
    number: int = Field(..., examples=[42, 7, 100])


print(response_format_from_pydantic_model(Pass1))
print(response_format_from_pydantic_model(Pass2))
print(response_format_from_pydantic_model(Failed))

The result :

uv run /home/.../PycharmProjects/.../.venv/bin/python /home/.../PycharmProjects/.../benchmarks/format.py 
type='json_schema' json_schema=JSONSchema(name='Pass1', schema_definition={'properties': {'string': {'title': 'String', 'type': 'integer'}, 'boolean': {'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass1', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
type='json_schema' json_schema=JSONSchema(name='Pass2', schema_definition={'properties': {'string': {'examples': ['example1', 'example2'], 'title': 'String', 'type': 'integer'}, 'boolean': {'examples': [True, False], 'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass2', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
Traceback (most recent call last):
  File "/home/.../PycharmProjects/.../benchmarks/format.py", line 24, in <module>
    print(response_format_from_pydantic_model(Failed))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/response_format.py", line 13, in response_format_from_pydantic_model
    model_schema = rec_strict_json_schema(model.model_json_schema())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 15, in rec_strict_json_schema
    schema_node[key] = rec_strict_json_schema(value)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 15, in rec_strict_json_schema
    schema_node[key] = rec_strict_json_schema(value)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 15, in rec_strict_json_schema
    schema_node[key] = rec_strict_json_schema(value)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 18, in rec_strict_json_schema
    schema_node[i] = rec_strict_json_schema(value)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 20, in rec_strict_json_schema
    raise ValueError(f"Unexpected type: {schema_node}")
ValueError: Unexpected type: 42

Process finished with exit code 1

The result after patchfix :

uv run /home/.../PycharmProjects/.../.venv/bin/python /home/.../PycharmProjects/.../benchmarks/format.py 
type='json_schema' json_schema=JSONSchema(name='Pass1', schema_definition={'properties': {'string': {'title': 'String', 'type': 'integer'}, 'boolean': {'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass1', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
type='json_schema' json_schema=JSONSchema(name='Pass2', schema_definition={'properties': {'string': {'examples': ['example1', 'example2'], 'title': 'String', 'type': 'integer'}, 'boolean': {'examples': [True, False], 'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass2', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
type='json_schema' json_schema=JSONSchema(name='Failed', schema_definition={'properties': {'string': {'examples': ['example1', 'example2'], 'title': 'String', 'type': 'integer'}, 'boolean': {'examples': [True, False], 'title': 'Boolean', 'type': 'boolean'}, 'number': {'examples': [42, 7, 100], 'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Failed', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)

Process finished with exit code 0

I use pydantic models and the examples attribute of the Field descriptor in my structured output projects whenever possible. I noticed in the Mistral documentation that there was a function for obtaining a schema, and I wanted to test it to compare the one I had made with yours, and I noticed that I had a strange error.

It seems that the `int` was omitted from what I assume to be the list of primary types in the `rec_strict_json_schema` function.

Here's how to reproduce the error:
```python
from mistralai.extra import response_format_from_pydantic_model
from pydantic import BaseModel, Field


class Pass1(BaseModel):
    string: int = Field(...)
    boolean: bool = Field(...)
    number: int = Field(...)

class Pass2(BaseModel):
    string: int = Field(..., examples=["example1", "example2"])
    boolean: bool = Field(..., examples=[True, False])
    number: int = Field(...)


class Failed(BaseModel):
    string: int = Field(..., examples=["example1", "example2"])
    boolean: bool = Field(..., examples=[True, False])
    number: int = Field(..., examples=[42, 7, 100])


print(response_format_from_pydantic_model(Pass1))
print(response_format_from_pydantic_model(Pass2))
print(response_format_from_pydantic_model(Failed))
```

The result :
```
uv run /home/.../PycharmProjects/.../.venv/bin/python /home/.../PycharmProjects/.../benchmarks/format.py 
type='json_schema' json_schema=JSONSchema(name='Pass1', schema_definition={'properties': {'string': {'title': 'String', 'type': 'integer'}, 'boolean': {'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass1', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
type='json_schema' json_schema=JSONSchema(name='Pass2', schema_definition={'properties': {'string': {'examples': ['example1', 'example2'], 'title': 'String', 'type': 'integer'}, 'boolean': {'examples': [True, False], 'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass2', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
Traceback (most recent call last):
  File "/home/.../PycharmProjects/.../benchmarks/format.py", line 24, in <module>
    print(response_format_from_pydantic_model(Failed))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/response_format.py", line 13, in response_format_from_pydantic_model
    model_schema = rec_strict_json_schema(model.model_json_schema())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 15, in rec_strict_json_schema
    schema_node[key] = rec_strict_json_schema(value)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 15, in rec_strict_json_schema
    schema_node[key] = rec_strict_json_schema(value)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 15, in rec_strict_json_schema
    schema_node[key] = rec_strict_json_schema(value)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 18, in rec_strict_json_schema
    schema_node[i] = rec_strict_json_schema(value)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../PycharmProjects/.../.venv/lib/python3.12/site-packages/mistralai/extra/utils/_pydantic_helper.py", line 20, in rec_strict_json_schema
    raise ValueError(f"Unexpected type: {schema_node}")
ValueError: Unexpected type: 42

Process finished with exit code 1
```

The result after patchfix :
```
uv run /home/.../PycharmProjects/.../.venv/bin/python /home/.../PycharmProjects/.../benchmarks/format.py 
type='json_schema' json_schema=JSONSchema(name='Pass1', schema_definition={'properties': {'string': {'title': 'String', 'type': 'integer'}, 'boolean': {'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass1', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
type='json_schema' json_schema=JSONSchema(name='Pass2', schema_definition={'properties': {'string': {'examples': ['example1', 'example2'], 'title': 'String', 'type': 'integer'}, 'boolean': {'examples': [True, False], 'title': 'Boolean', 'type': 'boolean'}, 'number': {'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Pass2', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)
type='json_schema' json_schema=JSONSchema(name='Failed', schema_definition={'properties': {'string': {'examples': ['example1', 'example2'], 'title': 'String', 'type': 'integer'}, 'boolean': {'examples': [True, False], 'title': 'Boolean', 'type': 'boolean'}, 'number': {'examples': [42, 7, 100], 'title': 'Number', 'type': 'integer'}}, 'required': ['string', 'boolean', 'number'], 'title': 'Failed', 'type': 'object', 'additionalProperties': False}, description=Unset(), strict=True)

Process finished with exit code 0
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant