AutoGen: Code Execution Issue in Multi-Agent System

[Laxmi530]

I am using AutoGen to implement a multi-agent system for my inventory management task. The system integrates Python code for managing inventory databases. However, I am encountering an issue where the agents fail to execute code as expected. I tried with function calling and tools but there are no changes. I am not getting the expected output.
LLM - Qwen2.5-1.5B-Instruct
AutoGen - 0.7.1
Python - 3.12.7
The following is the code I tried.

from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager, register_function
from autogen.coding import LocalCommandLineCodeExecutor
from SQL_DB import inventry_check
from pathlib import Path

chat_llm_config_local = [{"model": "Qwen2.5-1.5B-Instruct", "api_key": "None", "base_url": "http://127.0.0.1:8000/v1"}]
chat_llm_config = {"temperature": 0.2, "seed": 42, "config_list": chat_llm_config_local}

INVENTRY_PROMPT = """ 
You are an intelligent agent tasked with checking the stock levels in the inventory database. 
Follow these instructions precisely:
1. Execute only the provided function to check the current stock levels of items. 
2. Avoid any external assumptions or knowledge outside the function.
3. Output the results of the function exactly as returned, without modifications or formatting changes.
"""

dir = Path("code")
dir.mkdir(exist_ok=True)
executor = LocalCommandLineCodeExecutor(work_dir=dir)

inventry_assistant = AssistantAgent(
    name="Inventry_Assistant",
    system_message=INVENTRY_PROMPT,
    llm_config=chat_llm_config,
    code_execution_config={"executor": executor}
)

user_proxy = UserProxyAgent(
    name = "user_proxy",
    human_input_mode = "NEVER",
    max_consecutive_auto_reply=5,
    is_termination_msg=lambda x:x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={"work_dir":"test", "use_docker": False},
    llm_config=chat_llm_config,
    system_message="""Reply TERMINATE if the task has been solved at full satisfication Otherwise, reply CONTINUE, or the reason why the task is not solved yet."""
    )
# Method 1
inventry_assistant.register_for_llm(name="inventry_check", description="Get item, quantity and unit price from the inventry database.")(inventry_check)
user_proxy.register_for_execution(name="inventry_check")(inventry_check)
print(user_proxy.initiate_chat(inventry_assistant, message="How many item left in the inventry?"))

# Method 2
register_function(
    inventry_check,
    caller=inventry_assistant,
    executor=user_proxy,
    name="inventry_check",
    description="To check the inventry database.",
    )
print(user_proxy.initiate_chat(inventry_assistant, "How many item left in the inventry?"))
# Method 3
# Function for the inventory check
from sqlalchemy import create_engine, Table, MetaData, select
from sqlalchemy.orm import sessionmaker
from typing import List, Dict

@user_proxy.register_for_execution()
@inventry_assistant.register_for_llm(description="Get item, unit and unit price from the inventry database.")
def inventry_check() ->List[Dict]:
    """
    Connects to inventry DB, retrieves items with low quantities based on the threshold.
    :return: List of dictionaries containing item details.
    """
    engine = create_engine(inventry_db_url, echo=True)
    metadata = MetaData()
    ItemTable = Table("Inventry", metadata, autoload_with=engine)
    Session = sessionmaker(bind=engine)
    session = Session()
    try:
        result = session.execute(select(ItemTable).where(ItemTable.c.quantity <= ItemTable.c.threshold)).fetchall()
        items = [{"Item": row[1], "Quantity": row[2], "Unit price": row[3], "Threshold": 10} for row in result]
    finally:
        session.close()
    return items
print(user_proxy.initiate_chat(inventry_assistant, message="How many mobile left in the inventry?"))

I tried with different prompt and tried with Llama 3.2 1B model but there are no changes.
Can someone please guide me?

Thanking you in advance.

[ekzhu]

Our current stable version is v0.4.3, you are not using a Microsoft-published package. Please install autogen-agentchat.

Also, your code is using v0.2 API, please see the migration guide for v0.4: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/migration-guide.html

For your example, I tried with llama3.2 2B using Ollama:

import asyncio
from typing import Dict, List
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

def inventry_check() -> List[Dict]:
    """Get item, unit and unit price from the inventory database."""
    result = [
        (1, "Apple", 5, 10),
        (2, "Banana", 15, 5),
        (3, "Orange", 7, 8),
        (4, "Pineapple", 20, 15),
        (5, "Mango", 10, 20),
    ]
    items = [{"Item": row[1], "Quantity": row[2], "Unit price": row[3]} for row in result]
    return items

async def main() -> None:
    model_client = OpenAIChatCompletionClient(
        model="llama3.2:latest",
        base_url="http://localhost:11434/v1",
        api_key="placeholder",
        model_info={
            "vision": False,
            "function_calling": True,
            "json_output": False,
            "family": "unknown",
        },
    )
    agent = AssistantAgent(
        "assistant",
        model_client=model_client,
        tools=[inventry_check],
        reflect_on_tool_use=True,
    )
    stream = agent.run_stream(task="How many apples are in the inventory?")
    await Console(stream)


asyncio.run(main())

Output:

---------- user ----------
How many apples are in the inventory?
---------- assistant ----------
[FunctionCall(id='call_xzaeoawp', arguments='{"item":"apple"}', name='inventry_check')]
---------- assistant ----------
[FunctionExecutionResult(content="[{'Item': 'Apple', 'Quantity': 5, 'Unit price': 10}, {'Item': 'Banana', 'Quantity': 15, 'Unit price': 5}, {'Item': 'Orange', 'Quantity': 7, 'Unit price': 8}, {'Item': 'Pineapple', 'Quantity': 20, 'Unit price': 15}, {'Item': 'Mango', 'Quantity': 10, 'Unit price': 20}]", call_id='call_xzaeoawp')]
---------- assistant ----------
The quantity of apples in the inventory is 5.

[Laxmi530]

Thanks @ekzhu for the help.
Have few more questions

1. I am trying to use mistral model(mistral-large-latest) through API but I am getting error same API is working with openAI. Below is the error.
   UnprocessableEntityError: Error code: 422 - {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'messages', 0, 'user', 'name'], 'msg': 'Extra inputs are not permitted', 'input': 'user'}]}
2. Is there an option to terminate the program under certain conditions? For example, if I create an agent to check my email for a specific item and it does not find it, can the program be terminated?
3. Can an agent perform a task iteratively, such as checking the email once every three minutes for a total of three times?
4. Suppose I have 5 agents, and based on the information from the 2nd agent, I want to skip the 3rd and 4th agents. This means that, based on certain conditions, the information from the 2nd agent will be used by the 5th agent. How can we do this?

Your guidance will be very helpful for me
Thanking you in advance.

[ekzhu]

1. Is mistral API open AI compatible? If not it cannot be used with OpenAIChatCompletionClient. We will need to use Semantic Kernel Model adapter or a proxy server
2. You can run your agent 1 step at a time for better control. If you are using a Team, you can use termination conditions.
3. This should be done by the application. I am sure there are many existing tools can do this, e.g., cron jobs.
4. Using AgentChat, you can use SelectorGroupChat ith custom selector_func. For the Core API, the communication topology is 100% under your control so your application will be responsible for it.

[Laxmi530]

Thanks @ekzhu for the help.

1. I have attached the screenshot of using mistral API using openAI. Can you please guide me further steps on how to use the mistral API in autogen using Semantic Kernel Model adapter or a proxy server.

And point 2, 3, and 4 will try and if I need any clarification will post here.

Thanking you in advance.

[ekzhu]

Try add a "name" field in your message and set any value for the name field.

The name field is supported by OpenAI to differentiate between different authors of the message.

You can read about the semantic kernel model adapter in API docs. https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.models.semantic_kernel.html#autogen_ext.models.semantic_kernel.SKChatCompletionAdapter

For proxy server you may look at LiteLLM.