Skip to content

README.md

Latest commit

 

History

History
49 lines (31 loc) · 3.17 KB

README.md

File metadata and controls

49 lines (31 loc) · 3.17 KB
Raw

Of course. Here is a tutorial based on the content of the file.

A Guide to Evaluation and Tracing in the Microsoft Agent Framework

This tutorial will guide you through the tools available for evaluating, tracing, and debugging your agents within the Microsoft Agent Framework. A key part of developing robust AI agents is understanding their behavior, and these tools provide the necessary insights.

We will cover two main components:

  1. DevUI: A web-based user interface for real-time visualization and debugging of agent sessions.
  2. Observability: How to configure logging to trace the step-by-step execution of your agents.

1. DevUI: Visualizing Agent Interactions

The DevUI is a powerful web-based tool designed to give you a clear, real-time view into your agent's inner workings. It helps you visualize the entire interaction flow, from the initial prompt to the final response, including the agent's thought process and any tools it utilizes.

Key Features:

  • Session Visualization: Track the conversation and agent activities as they happen.
  • Debugging: Easily inspect the messages, tool calls, and LLM responses to understand why an agent is behaving in a certain way.
  • Interactive Interface: A user-friendly view that is much easier to parse than raw console logs.

How to Get Started:

To use the DevUI, you typically run a command provided by the agent framework's command-line interface. This command starts a local web server, allowing you to connect and monitor your agent sessions through your browser.

Sample Code:

For more detailed information and setup instructions, please visit the official DevUI package page: https://github.com/microsoft/agent-framework/tree/main/python/packages/devui

2. Observability: Logging and Tracing Agent Execution

While the DevUI is excellent for real-time visual debugging, observability through logging provides a persistent and detailed text-based record of an agent's execution. The framework is built to integrate with standard Python logging, making it easy to capture the data you need for analysis.

What You Can Trace:

  • User Requests: The initial input that starts the agent's task.
  • Agent's "Thoughts": The internal reasoning steps the agent takes.
  • Tool Invocations: Which tools the agent decides to use and with what parameters.
  • Final Responses: The output generated by the agent.

How it Works:

The framework automatically emits events during an agent's run. By configuring a logger, you can direct these events to your console or a file. The provided samples demonstrate how to set up this logging to see a sequential trace of the agent's operations, giving you a complete picture for post-execution analysis and debugging.

To review the sample code and learn how to configure the logger, check out the observability example here: tracer_aspire