Python: ChatCompletionAgent to produce structured output with different schemas

[vslepakov]

Hey everyone,

I'm using a ChatCompletionAgent with multiple plugins, like [WeatherPlugin(), MenuPlugin()], where each plugin contains multiple kernel functions.

What I'm aiming for is to have agent.get_response() return a structured output that includes both the natural language response and the structured data returned by the kernel function. Ideally, the result would look like this:

{
  "natural_text_response": "lorem ipsum...",
  "structured_response": <JSON object or string representation of the function's return value>
}

The challenge I’m facing is that the only mechanism I’ve found to define an output format is through AzureChatPromptExecutionSettings, which requires specifying a concrete output schema. This doesn’t seem to support dynamic return schemas across different functions with varying return types.

Is there a way to achieve this goal?

[sophialagerkranspandey]

Yes, would recommend checking out the docs here: https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/function-invocation?pivots=programming-language-python

[vslepakov]

Thanks! Not sure I am following. Are you referring to FunctionResultContent?

[moonbox3]

@vslepakov today, we don't have any support for this in SK Python. @markwallace-microsoft, do you know if there is any support for this in SK .NET that we could use as a guide/foundation? Thanks.