How to specify Agent without requiring structured output?

[davidbernat]

Question

Simple question. Unclear why the docs are not helping me resolve this inline so here I am.

I want to use Agent to generate a general purpose string chat response. The system I encoded works as intended when output_type is a BaseModel. But what about when all I want is any generated string? I noticed that the default for output_type=str, but that, in my understanding, actually still does the PydanticAI implementation under the hood of requiring the LLM to call the final_result(parameter: str) tool, which is additional overhead; and I noticed that the LLM fails at this task (because we are testing with small GPU models). Passing None did not function as direct chat, and the code agents with Context7 and e.g. Google/Bing provide differing results, none of which are the solution for this particular pattern. (For instance, Gemini said to use output_type=Agent[None | str], but see above.) What is the correct solution?

        llm = OpenAIChatModel(model_name=model_name, provider=provider)
        return Agent(llm, output_type=output_type,
                     instructions=instructions or "Be concise, reply with the correct answer.",
                     event_stream_handler=handler,
                     instrument=InstrumentationSettings(include_content=True))

Not blocker. Just inconvenient. Thanks.

Additional Context

No response

[davidbernat]

For those who arrive to this ticket by search the answer is simple, but somewhat opaque by the backend math of Pydantic.

In short:

1. omit output_type= and the Agent will accept any string response, i.e. default chat behavior
2. set output_type=str and the Agent will create the usual overhead of structured responses which, in this case, is to call a tool with a str argument which contains the string response; and, a generic non-tool use string will also be accepted. Taken together these seem to provide a clear, unnecessary overhead, which actually did fail to achieve a result on occasion.
3. set output_type=None is simply wrong, and the Agent will attempt to data munge a response to None, and fail repeatedly.

And:

1. The clear answer is to omit output_type= from the Agent constructor.
2. The output_type=str exists to allow the Agent to respond with "just text" when other output types are also included, i.e., you will most likely specify str when also provided other return types in union, i.e., output_type= CityState | str, in which case the run ends either by calling the CityState function tool or the str function tool or simply returning text with no tool use.

Hope this helps who needs this. This ticket can now be closed.

Here are the references:

Where Documentation Explains the Difference

1. How runs end (top of output.md):

When output_type is not specified, any plain text response from the model becomes the output. The documentation states:

"A run ends when the model responds with one of the output types, or, if no output type is specified or str is one of the allowed options, when a plain text response is received."
— output.md, line 13

2. When str IS vs. ISN'T in output types (Structured Output section):

"If str is not among the output types, the model is forced to return structured data or call an output function."
— output.md, "Structured output" section

This means: when str IS included, the model can choose plain text, but it's still presented as an output tool option the model must be aware of.

3. How str-like outputs work by default (Text Output section):

"If you provide an output function that takes a string, Pydantic AI will by default create an output tool like for any other output function."
— output.md, "Text output" section

This implies that even pure str as output_type is treated as an output tool with associated overhead.

[DouweM]

2. set output_type=str and the Agent will create the usual overhead of structured responses which, in this case, is to call a tool with a str argument which contains the string response

@davidbernat Are you sure? output_type=str should behave exactly the same as omitting output_type, which should let the model generate text the regular way, without us building any output tools.

[davidbernat]

@DouweM Not conclusively. I only read the documentation and the cursory experiments, not an in-depth look into your code.

In response to your comment, I re-ran what I had been using as my baseline, now using the HTTPx logger that I had written for PydanticAI previously to see the requests going to the LLM, in order to directly confirm that the arguments function was added. In the case that output_str=str what I see now is 1. the coercion function is not added (your point above) and 2. there is a model_alteration prompt output that does appear to be added: model_alterations prompted_output_template= Always respond with a JSON object that's compatible with this schema: {schema} Don't include any text or Markdown fencing before or after. Using output_str=str does retry every once in a while for reasons which are not related to timeout, but I do not have any insight into why. Do you have something you want to say about the model_alterations component that does or does not impact its workings? The overall sense I had originally, and your comment asserts, is that output_str=str should be treated as the "chat" model. I just did not see that playing out in every use and removing output_str= works slightly more naturally.

More or less your statement is correct, and my comments above should be amended. Thanks.