How to use code execution in agents

[raagul-qb]

What happened?

I executed the following code

async def execute_code(code_block:str) -> str:
    local_executor = LocalCommandLineCodeExecutor(work_dir=work_dir)
    code_executor_agent = CodeExecutorAgent("code_executor", code_executor=local_executor)
    # Run the agent with a given code snippet.
    task = TextMessage(
        content=code_block,
        source="user",
    )
    response = await code_executor_agent.on_messages([task], CancellationToken())
    print(response.chat_message)
    return response.chat_message.content
Admin = AssistantAgent(
    "Admin",
    model_client=model_client,
    handoffs=["coder","user"],
    system_message="""You are a system admin who assists the user.
                      Coordinate Development by delegating to specialized agents:
                      coder - If the user asks to write and execute code had over to coder
                      If you need information from the user, you must first send your message, then you can handoff to the user.
                      Always send your plan first, then handoff to appropriate agent.
                      Always handoff to a single agent at a time.
                      Use TERMINATE when research is complete.
                    """,
)
coder = AssistantAgent(
    name="coder",
    system_message=""" Write code for the query user gives always start your code with ```.
    Only write python code and nothing else. 
    use execute_code tool to execute code user send by sending in the code to it.
    """,
    model_client=model_client,
    handoffs=["Admin"],
    tools=[execute_code]
)
termination = HandoffTermination(target="user") | TextMentionTermination("TERMINATE")
team = Swarm([Admin,coder], termination_condition=termination)
async def run_team_stream() -> None:
    task = input("Enter a task:")
    task_result = await Console(team.run_stream(task=task))
    last_message = task_result.messages[-1]

    while isinstance(last_message, HandoffMessage) and last_message.target == "user":
        user_message = input("User: ")

        task_result = await Console(
            team.run_stream(task=HandoffMessage(source="user", target=last_message.source, content=user_message))
        )
        last_message = task_result.messages[-1]
await run_team_stream()

It does not execute the code properly and outputs the

following

Enter a task:Write a code to multiply 2 matrices
---------- user ----------
Write a code to multiply 2 matrices
---------- Admin ----------
[FunctionCall(id='call_D9HjRhpJ8OOYAMwfbLqZyVOc', arguments='{}', name='transfer_to_coder')]
---------- Admin ----------
[FunctionExecutionResult(content='Transferred to coder, adopting the role of coder immediately.', call_id='call_D9HjRhpJ8OOYAMwfbLqZyVOc', is_error=False)]
---------- Admin ----------
Transferred to coder, adopting the role of coder immediately.
/usr/local/lib/python3.11/dist-packages/autogen_agentchat/agents/_assistant_agent.py:415: UserWarning: Both tool_calls and content are present in the message. This is unexpected. content will be ignored, tool_calls will be used.
  model_result = await self._model_client.create(
---------- coder ----------
```python
import numpy as np

def multiply_matrices(matrix1, matrix2):
    # Convert lists to numpy arrays
    matrix1 = np.array(matrix1)
    matrix2 = np.array(matrix2)

    # Check if multiplication is possible
    if matrix1.shape[1] != matrix2.shape[0]:
        return "Multiplication not possible. Number of columns in first matrix should be equal to number of rows in second matrix."

    # Perform multiplication
    result = np.dot(matrix1, matrix2)

    # Convert numpy array back to list
    result = result.tolist()

    return result

# Test the function
matrix1 = [[1, 2, 3], [4, 5, 6]]
matrix2 = [[7, 8], [9, 10], [11, 12]]
print(multiply_matrices(matrix1, matrix2))

---------- coder ----------
[FunctionCall(id='call_xsezME1MZOs4RYBjUz7RXF8v', arguments='{\n"code_block": "\nimport numpy as np\n\ndef multiply_matrices(matrix1, matrix2):\n    # Convert lists to numpy arrays\n    matrix1 = np.array(matrix1)\n    matrix2 = np.array(matrix2)\n\n    # Check if multiplication is possible\n    if matrix1.shape[1] != matrix2.shape[0]:\n        return \'Multiplication not possible. Number of columns in first matrix should be equal to number of rows in second matrix.\'\n\n    # Perform multiplication\n    result = np.dot(matrix1, matrix2)\n\n    # Convert numpy array back to list\n    result = result.tolist()\n\n    return result\n\n# Test the function\nmatrix1 = [[1, 2, 3], [4, 5, 6]]\nmatrix2 = [[7, 8], [9, 10], [11, 12]]\nmultiply_matrices(matrix1, matrix2)\n"\n}', name='execute_code')]
---------- coder ----------
[FunctionExecutionResult(content='Error: Invalid control character at: line 2 column 16 (char 17)', call_id='call_xsezME1MZOs4RYBjUz7RXF8v', is_error=True)]
---------- coder ----------
Error: Invalid control character at: line 2 column 16 (char 17)
source='code_executor' models_usage=None content='No code blocks found in the thread. Please provide at least one markdown-encoded code block to execute (i.e., quoting code in ```python or ```sh code blocks).' type='TextMessage'
---------- coder ----------
[FunctionCall(id='call_oe1xCUYLN5ELJ0BFNYmnIRxz', arguments='{\n"code_block": "import numpy as np\\n\\ndef multiply_matrices(matrix1, matrix2):\\n    # Convert lists to numpy arrays\\n    matrix1 = np.array(matrix1)\\n    matrix2 = np.array(matrix2)\\n\\n    # Check if multiplication is possible\\n    if matrix1.shape[1] != matrix2.shape[0]:\\n        return \'Multiplication not possible. Number of columns in first matrix should be equal to number of rows in second matrix.\'\\n\\n    # Perform multiplication\\n    result = np.dot(matrix1, matrix2)\\n\\n    # Convert numpy array back to list\\n    result = result.tolist()\\n\\n    return result\\n\\n# Test the function\\nmatrix1 = [[1, 2, 3], [4, 5, 6]]\\nmatrix2 = [[7, 8], [9, 10], [11, 12]]\\nmultiply_matrices(matrix1, matrix2)"\n}', name='execute_code')]
---------- coder ----------
[FunctionExecutionResult(content='No code blocks found in the thread. Please provide at least one markdown-encoded code block to execute (i.e., quoting code in ```python or ```sh code blocks).', call_id='call_oe1xCUYLN5ELJ0BFNYmnIRxz', is_error=False)]
---------- coder ----------
No code blocks found in the thread. Please provide at least one markdown-encoded code block to execute (i.e., quoting code in ```python or ```sh code blocks).
source='code_executor' models_usage=None content='No code blocks found in the thread. Please provide at least one markdown-encoded code block to execute (i.e., quoting code in ```python or ```sh code blocks).' type='TextMessage'

What did you expect to happen?

Code should be executed properly

How can we reproduce it (as minimally and precisely as possible)?

async def execute_code(code_block:str) -> str:
    local_executor = LocalCommandLineCodeExecutor(work_dir=work_dir)
    code_executor_agent = CodeExecutorAgent("code_executor", code_executor=local_executor)
    # Run the agent with a given code snippet.
    task = TextMessage(
        content=code_block,
        source="user",
    )
    response = await code_executor_agent.on_messages([task], CancellationToken())
    print(response.chat_message)
    return response.chat_message.content
Admin = AssistantAgent(
    "Admin",
    model_client=model_client,
    handoffs=["coder","user"],
    system_message="""You are a system admin who assists the user.
                      Coordinate Development by delegating to specialized agents:
                      coder - If the user asks to write and execute code had over to coder
                      If you need information from the user, you must first send your message, then you can handoff to the user.
                      Always send your plan first, then handoff to appropriate agent.
                      Always handoff to a single agent at a time.
                      Use TERMINATE when research is complete.
                    """,
)
coder = AssistantAgent(
    name="coder",
    system_message=""" Write code for the query user gives always start your code with ```.
    Only write python code and nothing else. 
    use execute_code tool to execute code user send by sending in the code to it.
    """,
    model_client=model_client,
    handoffs=["Admin"],
    tools=[execute_code]
)
termination = HandoffTermination(target="user") | TextMentionTermination("TERMINATE")
team = Swarm([Admin,coder], termination_condition=termination)
async def run_team_stream() -> None:
    task = input("Enter a task:")
    task_result = await Console(team.run_stream(task=task))
    last_message = task_result.messages[-1]

    while isinstance(last_message, HandoffMessage) and last_message.target == "user":
        user_message = input("User: ")

        task_result = await Console(
            team.run_stream(task=HandoffMessage(source="user", target=last_message.source, content=user_message))
        )
        last_message = task_result.messages[-1]
await run_team_stream()

AutoGen version

0.4

Which package was this bug in

AgentChat

Model used

No response

Python version

No response

Operating system

No response

Any additional info you think would be helpful for fixing this bug

No response

[ekzhu]

Could you please format the code? It's difficult to read.

use

```python
```

[raagul-qb]

I apologize for the poor formatting on my side. I have fixed it now

[ekzhu]

Instead of using a CodeExecutorAgent and wraps it inside a tool, use the PythonCodeExecutionTool: https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.tools.code_execution.html#autogen_ext.tools.code_execution.PythonCodeExecutionTool

In your output:

---------- coder ----------
[FunctionCall(id='call_xsezME1MZOs4RYBjUz7RXF8v', arguments='{\n"code_block": "\nimport numpy as np\n\ndef multiply_matrices(matrix1, matrix2):\n    # Convert lists to numpy arrays\n    matrix1 = np.array(matrix1)\n    matrix2 = np.array(matrix2)\n\n    # Check if multiplication is possible\n    if matrix1.shape[1] != matrix2.shape[0]:\n        return \'Multiplication not possible. Number of columns in first matrix should be equal to number of rows in second matrix.\'\n\n    # Perform multiplication\n    result = np.dot(matrix1, matrix2)\n\n    # Convert numpy array back to list\n    result = result.tolist()\n\n    return result\n\n# Test the function\nmatrix1 = [[1, 2, 3], [4, 5, 6]]\nmatrix2 = [[7, 8], [9, 10], [11, 12]]\nmultiply_matrices(matrix1, matrix2)\n"\n}', name='execute_code')]
---------- coder ----------
[FunctionExecutionResult(content='Error: Invalid control character at: line 2 column 16 (char 17)', call_id='call_xsezME1MZOs4RYBjUz7RXF8v', is_error=True)]
---------- coder ----------
Error: Invalid control character at: line 2 column 16 (char 17)
source='code_executor' models_usage=None content='No code blocks found in the thread. Please provide at least one markdown-encoded code block to execute (i.e., quoting code in ```python or ```sh code blocks).' type='TextMessage'

Looking at the first output, you can see the input to the function does not have code blocks

```python
```

around the code so the code executor agent cannot detect code.

The CodeExecutorAgent only detects code if the input is in markdown format.