bug: Warn about Azure OpenAI completions file-incompatibility

docs/input.md

@@ -110,7 +110,7 @@ Support for file URLs varies depending on type and provider:
 
 | Model | Send URL directly | Download and send bytes | Unsupported |
 |-------|-------------------|-------------------------|-------------|
-| [`OpenAIChatModel`][pydantic_ai.models.openai.OpenAIChatModel] | `ImageUrl` | `AudioUrl`, `DocumentUrl` | `VideoUrl` |
+| [`OpenAIChatModel`][pydantic_ai.models.openai.OpenAIChatModel] | `ImageUrl` | `AudioUrl`, `DocumentUrl`* | `VideoUrl` |
 | [`OpenAIResponsesModel`][pydantic_ai.models.openai.OpenAIResponsesModel] | `ImageUrl`, `AudioUrl`, `DocumentUrl` | — | `VideoUrl` |
 | [`AnthropicModel`][pydantic_ai.models.anthropic.AnthropicModel] | `ImageUrl`, `DocumentUrl` (PDF) | `DocumentUrl` (`text/plain`) | `AudioUrl`, `VideoUrl` |
 | [`GoogleModel`][pydantic_ai.models.google.GoogleModel] (Vertex) | All URL types | — | — |
@@ -120,7 +120,11 @@ Support for file URLs varies depending on type and provider:
 | [`BedrockConverseModel`][pydantic_ai.models.bedrock.BedrockConverseModel] | S3 URLs (`s3://`) | `ImageUrl`, `DocumentUrl`, `VideoUrl` | `AudioUrl` |
 | [`OpenRouterModel`][pydantic_ai.models.openrouter.OpenRouterModel] | `ImageUrl`, `DocumentUrl` | `AudioUrl` | `VideoUrl` |
 
-A model API may be unable to download a file (e.g., because of crawling or access restrictions) even if it supports file URLs. For example, [`GoogleModel`][pydantic_ai.models.google.GoogleModel] on Vertex AI limits YouTube video URLs to one URL per request. In such cases, you can instruct Pydantic AI to download the file content locally and send that instead of the URL by setting `force_download` on the URL object:
+*Not supported with `AzureProvider`. Use [`OpenAIResponsesModel` with `AzureProvider`](models/openai.md#using-azure-with-the-responses-api) instead.
+
+A model API may be unable to download a file (e.g., because of crawling or access restrictions) even if it supports file URLs. For example, [`GoogleModel`][pydantic_ai.models.google.GoogleModel] on Vertex AI limits YouTube video URLs to one URL per request.
+
+In such cases, you can instruct Pydantic AI to download the file content locally and send that instead of the URL by setting `force_download` on the URL object:
 
 ```py {title="force_download.py" test="skip" lint="skip"}
 from pydantic_ai import ImageUrl, AudioUrl, VideoUrl, DocumentUrl

docs/models/openai.md

@@ -427,6 +427,32 @@ agent = Agent(model)
 ...
 ```
 
+#### Using Azure with the Responses API
+
+Azure AI Foundry also supports the OpenAI Responses API through [`OpenAIResponsesModel`][pydantic_ai.models.openai.OpenAIResponsesModel]. This is particularly recommended when working with document inputs (`DocumentUrl` and `BinaryContent`), as Azure's Chat Completions API does not support these input types.
+
+??? example "Document processing with Azure using Responses API"
+    ```python
+    from pydantic_ai import Agent, BinaryContent
+    from pydantic_ai.models.openai import OpenAIResponsesModel
+    from pydantic_ai.providers.azure import AzureProvider
+
+    pdf_bytes = b'%PDF-1.4 ...'  # Your PDF content
+
+    model = OpenAIResponsesModel(
+        'gpt-5',
+        provider=AzureProvider(
+            azure_endpoint='your-azure-endpoint',
+            api_version='your-api-version',
+        ),
+    )
+    agent = Agent(model)
+    result = agent.run_sync([
+        'Summarize this document',
+        BinaryContent(data=pdf_bytes, media_type='application/pdf'),
+    ])
+    ```
+
 ### Vercel AI Gateway
 
 To use [Vercel's AI Gateway](https://vercel.com/docs/ai-gateway), first follow the [documentation](https://vercel.com/docs/ai-gateway) instructions on obtaining an API key or OIDC token.

learnings

@@ -0,0 +1 @@
+/Users/david/projects/pydantic-ai-claude-tools/learnings

pydantic_ai_slim/pydantic_ai/models/openai.py

@@ -13,7 +13,7 @@
 
 from pydantic import BaseModel, TypeAdapter, ValidationError
 from pydantic_core import to_json
-from typing_extensions import assert_never, deprecated
+from typing_extensions import Never, assert_never, deprecated
 
 from .. import ModelAPIError, ModelHTTPError, UnexpectedModelBehavior, _utils, usage
 from .._output import DEFAULT_OUTPUT_TOOL_NAME, OutputObjectDefinition
@@ -1184,6 +1184,8 @@ async def _map_user_prompt(self, part: UserPromptPart) -> chat.ChatCompletionUse
                             audio = InputAudio(data=item.base64, format=item.format)
                         content.append(ChatCompletionContentPartInputAudioParam(input_audio=audio, type='input_audio'))
                     elif item.is_document:
+                        if not profile.openai_chat_supports_document_input:
+                            self._raise_document_input_not_supported_error()
                         content.append(
                             File(
                                 file=FileFile(
@@ -1227,6 +1229,8 @@ async def _map_user_prompt(self, part: UserPromptPart) -> chat.ChatCompletionUse
                             )
                         )
                     else:
+                        if not profile.openai_chat_supports_document_input:
+                            self._raise_document_input_not_supported_error()
                         downloaded_item = await download_item(item, data_format='base64_uri', type_format='extension')
                         content.append(
                             File(
@@ -1237,7 +1241,7 @@ async def _map_user_prompt(self, part: UserPromptPart) -> chat.ChatCompletionUse
                                 type='file',
                             )
                         )
-                elif isinstance(item, VideoUrl):  # pragma: no cover
+                elif isinstance(item, VideoUrl):
                     raise NotImplementedError('VideoUrl is not supported for OpenAI')
                 elif isinstance(item, CachePoint):
                     # OpenAI doesn't support prompt caching via CachePoint, so we filter it out
@@ -1246,6 +1250,16 @@ async def _map_user_prompt(self, part: UserPromptPart) -> chat.ChatCompletionUse
                     assert_never(item)
         return chat.ChatCompletionUserMessageParam(role='user', content=content)
 
+    def _raise_document_input_not_supported_error(self) -> Never:
+        if self._provider.name == 'azure':
+            raise UserError(
+                "Azure's Chat Completions API does not support document input. "
+                'Use `OpenAIResponsesModel` with `AzureProvider` instead.'
+            )
+        raise UserError(
+            f'The {self._provider.name!r} provider does not support document input via the Chat Completions API.'
+        )
+
     @staticmethod
     def _is_text_like_media_type(media_type: str) -> bool:
         return (

pydantic_ai_slim/pydantic_ai/profiles/openai.py

@@ -118,6 +118,12 @@ class OpenAIModelProfile(ModelProfile):
     See https://github.com/pydantic/pydantic-ai/issues/3245 for more details.
     """
 
+    openai_chat_supports_document_input: bool = True
+    """Whether the Chat Completions API supports document content parts (type='file').
+
+    Some OpenAI-compatible providers (e.g. Azure) do not support document input via the Chat Completions API.
+    """
+
     def __post_init__(self):  # pragma: no cover
         if not self.openai_supports_sampling_settings:
             warnings.warn(

pydantic_ai_slim/pydantic_ai/providers/azure.py

@@ -67,10 +67,15 @@ def model_profile(self, model_name: str) -> ModelProfile | None:
 
                 # As AzureProvider is always used with OpenAIChatModel, which used to unconditionally use OpenAIJsonSchemaTransformer,
                 # we need to maintain that behavior unless json_schema_transformer is set explicitly
-                return OpenAIModelProfile(json_schema_transformer=OpenAIJsonSchemaTransformer).update(profile)
+                # Azure Chat Completions API doesn't support document input
+                return OpenAIModelProfile(
+                    json_schema_transformer=OpenAIJsonSchemaTransformer,
+                    openai_chat_supports_document_input=False,
+                ).update(profile)
 
         # OpenAI models are unprefixed
-        return openai_model_profile(model_name)
+        # Azure Chat Completions API doesn't support document input
+        return OpenAIModelProfile(openai_chat_supports_document_input=False).update(openai_model_profile(model_name))
 
     @overload
     def __init__(self, *, openai_client: AsyncAzureOpenAI) -> None: ...

tests/models/cassettes/test_openai/test_x_yaml_document_as_binary_content_input.yaml

@@ -0,0 +1,95 @@
+interactions:
+- request:
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, br
+      connection:
+      - keep-alive
+      content-length:
+      - '279'
+      content-type:
+      - application/json
+      host:
+      - api.openai.com
+    method: POST
+    parsed_body:
+      messages:
+      - content:
+        - text: What does this YAML describe?
+          type: text
+        - text: |-
+            -----BEGIN FILE id="a5bdf9" type="application/x-yaml"-----
+            name: test
+            version: 1.0.0
+            -----END FILE id="a5bdf9"-----
+          type: text
+        role: user
+      model: gpt-4o
+      stream: false
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    headers:
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      connection:
+      - keep-alive
+      content-length:
+      - '1880'
+      content-type:
+      - application/json
+      openai-organization:
+      - user-grnwlxd1653lxdzp921aoihz
+      openai-processing-ms:
+      - '4743'
+      openai-project:
+      - proj_FYsIItHHgnSPdHBVMzhNBWGa
+      openai-version:
+      - '2020-10-01'
+      strict-transport-security:
+      - max-age=31536000; includeSubDomains; preload
+      transfer-encoding:
+      - chunked
+    parsed_body:
+      choices:
+      - finish_reason: stop
+        index: 0
+        logprobs: null
+        message:
+          annotations: []
+          content: |-
+            The provided YAML snippet is a basic descriptor for something labeled with the name "test" and a version number "1.0.0". Without additional context or accompanying fields, it's difficult to definitively say what specific application or resource this is describing. In a general sense, such a YAML configuration could be used for various purposes, including but not limited to:
+
+            1. **Software/Application:** It could describe a software application or component called "test" with version 1.0.0.
+            2. **Configuration Management:** It might be a part of a configuration management system for managing different versions of a service or application.
+            3. **Package Information:** If used in a package management context, it might represent metadata for a package or library.
+            4. **Service Definition:** It could represent a service or microservice within a larger system.
+
+            Each of these interpretations would depend on the broader context in which this YAML file is used. Further fields in the YAML file would provide more specificity about its purpose and functionality.
+          refusal: null
+          role: assistant
+      created: 1769199521
+      id: chatcmpl-D1Hu5C2mqc2CPw07SQa6U7Ki9PF7X
+      model: gpt-4o-2024-08-06
+      object: chat.completion
+      service_tier: default
+      system_fingerprint: fp_deacdd5f6f
+      usage:
+        completion_tokens: 202
+        completion_tokens_details:
+          accepted_prediction_tokens: 0
+          audio_tokens: 0
+          reasoning_tokens: 0
+          rejected_prediction_tokens: 0
+        prompt_tokens: 57
+        prompt_tokens_details:
+          audio_tokens: 0
+          cached_tokens: 0
+        total_tokens: 259
+    status:
+      code: 200
+      message: OK
+version: 1

tests/models/cassettes/test_openai/test_yaml_document_as_binary_content_input.yaml

@@ -0,0 +1,92 @@
+interactions:
+- request:
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, br
+      connection:
+      - keep-alive
+      content-length:
+      - '308'
+      content-type:
+      - application/json
+      host:
+      - api.openai.com
+    method: POST
+    parsed_body:
+      messages:
+      - content:
+        - text: What type of configuration is this?
+          type: text
+        - text: |-
+            -----BEGIN FILE id="45a391" type="application/yaml"-----
+            version: "3"
+            services:
+              web:
+                image: nginx
+            -----END FILE id="45a391"-----
+          type: text
+        role: user
+      model: gpt-4o
+      stream: false
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    headers:
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      connection:
+      - keep-alive
+      content-length:
+      - '1218'
+      content-type:
+      - application/json
+      openai-organization:
+      - user-grnwlxd1653lxdzp921aoihz
+      openai-processing-ms:
+      - '1676'
+      openai-project:
+      - proj_FYsIItHHgnSPdHBVMzhNBWGa
+      openai-version:
+      - '2020-10-01'
+      strict-transport-security:
+      - max-age=31536000; includeSubDomains; preload
+      transfer-encoding:
+      - chunked
+    parsed_body:
+      choices:
+      - finish_reason: stop
+        index: 0
+        logprobs: null
+        message:
+          annotations: []
+          content: The configuration you provided is a YAML file for Docker Compose. Docker Compose is a tool used for defining
+            and running multi-container Docker applications. In this specific configuration, the YAML file is specifying a
+            single service called `web`, which uses the `nginx` Docker image. The file starts with specifying the Compose
+            file version as "3", indicating the format version used for composing the services.
+          refusal: null
+          role: assistant
+      created: 1769190655
+      id: chatcmpl-D1Fb52cAhS0I5T514KLWFLTvsJHYv
+      model: gpt-4o-2024-08-06
+      object: chat.completion
+      service_tier: default
+      system_fingerprint: fp_a0e9480a2f
+      usage:
+        completion_tokens: 77
+        completion_tokens_details:
+          accepted_prediction_tokens: 0
+          audio_tokens: 0
+          reasoning_tokens: 0
+          rejected_prediction_tokens: 0
+        prompt_tokens: 55
+        prompt_tokens_details:
+          audio_tokens: 0
+          cached_tokens: 0
+        total_tokens: 132
+    status:
+      code: 200
+      message: OK
+version: 1