notebook/agentchat_structured_outputs_from_config.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Structured output from json configuration\n",
    "\n",
    "Various LLM providers offer functionality for defining a structure of the messages generated by LLMs and AG2 enables this by propagating `response_format`, in the LLM configuration for your agents, to the underlying client.\n",
    "    \n",
    "You can define the JSON structure of the output in the `response_format` field in the LLM configuration.\n",
    "\n",
    "To assist in determining the JSON structure, you can generate a valid schema using `.model_json_schema()` on a predefined pydantic model, for more info, see [here](https://docs.pydantic.dev/latest/concepts/json_schema/). Your schema should be [OpenAPI specification](https://github.com/OAI/OpenAPI-Specification) compliant and have a **title** field defined for the root model which will be loaded as a `response_format` for the Agent.\n",
    "\n",
    "For more info on structured outputs, see [our documentation](https://docs.ag2.ai/docs/user-guide/basic-concepts/llm-configuration/structured-outputs).\n",
    "\n",
    "\n",
    "````{=mdx}\n",
    ":::info Requirements\n",
    "Install `ag2`:\n",
    "```bash\n",
    "pip install -U ag2[openai]\n",
    "```\n",
    "\n",
    "> **Note:** If you have been using `autogen` or `pyautogen`, all you need to do is upgrade it using:  \n",
    "> ```bash\n",
    "> pip install -U autogen\n",
    "> ```\n",
    "> or  \n",
    "> ```bash\n",
    "> pip install -U pyautogen\n",
    "> ```\n",
    "> as `pyautogen`, `autogen`, and `ag2` are aliases for the same PyPI package.  \n",
    "\n",
    "\n",
    "For more information, please refer to the [installation guide](https://docs.ag2.ai/docs/user-guide/basic-concepts/installing-ag2).\n",
    ":::\n",
    "````"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Supported clients\n",
    "AG2 has structured output support for following client providers:\n",
    "- OpenAI (`openai`)\n",
    "- Anthropic (`anthropic`)\n",
    "- Google Gemini (`google`)\n",
    "- Ollama (`ollama`)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set your API Endpoint\n",
    "\n",
    "The [`LLMConfig.from_json`](https://docs.ag2.ai/docs/api-reference/autogen/llm_config/LLMConfig) method loads a list of configurations from an environment variable or a JSON file.\n",
    "\n",
    "Here is an example of a configuration using the `gpt-4o-mini` model that will use a `MathReasoning` response format. To use it, paste it into your `OAI_CONFIG_LIST` file and set the `api_key` to your OpenAI API key.\n",
    "\n",
    "```json\n",
    "[\n",
    "    {\n",
    "        \"model\": \"gpt-4o-mini\",\n",
    "        \"api_key\": \"<YOUR_OPENAI_API_KEY>\",\n",
    "        \"response_format\": {\n",
    "            \"$defs\":{\n",
    "                \"Step\":{\n",
    "                    \"properties\":{\n",
    "                        \"explanation\":{\n",
    "                        \"title\":\"Explanation\",\n",
    "                        \"type\":\"string\"\n",
    "                        },\n",
    "                        \"output\":{\n",
    "                        \"title\":\"Output\",\n",
    "                        \"type\":\"string\"\n",
    "                        }\n",
    "                    },\n",
    "                    \"required\":[\n",
    "                        \"explanation\",\n",
    "                        \"output\"\n",
    "                    ],\n",
    "                    \"title\":\"Step\",\n",
    "                    \"type\":\"object\"\n",
    "                }\n",
    "            },\n",
    "            \"properties\":{\n",
    "                \"steps\":{\n",
    "                    \"items\":{\n",
    "                        \"$ref\":\"#/$defs/Step\"\n",
    "                    },\n",
    "                    \"title\":\"Steps\",\n",
    "                    \"type\":\"array\"\n",
    "                },\n",
    "                \"final_answer\":{\n",
    "                    \"title\":\"Final Answer\",\n",
    "                    \"type\":\"string\"\n",
    "                }\n",
    "            },\n",
    "            \"required\":[\n",
    "                \"steps\",\n",
    "                \"final_answer\"\n",
    "            ],\n",
    "            \"title\":\"MathReasoning\",\n",
    "            \"type\":\"object\"\n",
    "        },         \n",
    "        \"tags\": [\"gpt-4o-mini-response-format\"]\n",
    "    }\n",
    "]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import autogen\n",
    "\n",
    "# Load the configuration including the response format\n",
    "llm_config = autogen.LLMConfig.from_json(path=\"OAI_CONFIG_LIST\", cache_seed=42).where(tags=[\"gpt-4o-response-format\"])\n",
    "\n",
    "# Output the configuration, showing that it matches the configuration file.\n",
    "llm_config"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "````{=mdx}\n",
    ":::tip\n",
    "Learn more about configuring LLMs for agents [here](https://docs.ag2.ai/docs/user-guide/basic-concepts/llm-configuration/llm-configuration).\n",
    ":::\n",
    "````"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example: math reasoning\n",
    "\n",
    "Using structured output, we can enforce chain-of-thought reasoning in the model to output an answer in a structured, step-by-step way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define chat actors\n",
    "\n",
    "Now we can define the agents that will solve the posed math problem. \n",
    "We will keep this example simple; we will use a `UserProxyAgent` to input the math problem and an `AssistantAgent` to solve it.\n",
    "\n",
    "The `AssistantAgent` will be constrained to solving the math problem step-by-step by using the `MathReasoning` response format we defined above.\n",
    "\n",
    "The `response_format` is added to the LLM configuration and then this configuration is applied to the agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"User_proxy\",\n",
    "    system_message=\"A human admin.\",\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "\n",
    "assistant = autogen.AssistantAgent(\n",
    "    name=\"Math_solver\",\n",
    "    llm_config=llm_config,  # Response Format is in the configuration\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start the chat\n",
    "\n",
    "Let's now start the chat and prompt the assistant to solve a simple equation. The assistant agent should return a response solving the equation using a step-by-step `MathReasoning` model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary = user_proxy.initiate_chat(\n",
    "    assistant, message=\"how can I solve 8x + 7 = -23\", max_turns=1, summary_method=\"last_msg\"\n",
    ").summary\n",
    "\n",
    "summary"
   ]
  }
 ],
 "metadata": {
  "front_matter": {
   "description": "OpenAI offers a functionality for defining a structure of the messages generated by LLMs, AutoGen enables this functionality by propagating response_format passed to your agents to the underlying client.",
   "tags": [
    "structured output"
   ]
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}