.NET: Missing unified interface for AIAgent with memory, tools, and structured output

[napgab]

[.NET] Missing unified interface for AIAgent with memory, tools, and structured output

Problem Description

The Microsoft Agent Framework .NET implementation lacks a unified interface that allows developers to use memory, function tools, and structured output together with a single agent type. Currently, developers must choose between:

1. AIAgent: Supports memory (via AIContextProviderFactory) and tools, but lacks structured output (RunAsync<T>)
2. ChatClientAgent: Supports structured output (RunAsync<T>) but cannot use memory components

This architectural limitation forces developers into suboptimal solutions and prevents the framework from supporting common AI agent patterns that require all three capabilities simultaneously.

Current State Analysis

AIAgent (supports memory + tools, NO structured output)

Memory Example (Step13):

AIAgent agent = chatClient.CreateAIAgent(new ChatClientAgentOptions()
{
    Instructions = "You are a friendly assistant.",
    AIContextProviderFactory = ctx => new UserInfoMemory(
        chatClient.AsIChatClient(), 
        ctx.SerializedState, 
        ctx.JsonSerializerOptions)
});

// Only returns AgentRunResponse, not typed
AgentRunResponse response = await agent.RunAsync("Hello", thread);

Tools Example (Step03):

AIAgent agent = new AzureOpenAIClient(endpoint, credential)
    .GetChatClient(deploymentName)
    .CreateAIAgent(
        instructions: "You are a helpful assistant", 
        tools: [AIFunctionFactory.Create(GetWeather)]);

// Only returns AgentRunResponse, not typed  
AgentRunResponse response = await agent.RunAsync("What is the weather?");

ChatClientAgent (supports structured output, NO memory)

Structured Output Example (Step05):

ChatClientAgent agent = chatClient.CreateAIAgent(new ChatClientAgentOptions(
    name: "HelpfulAssistant", 
    instructions: "You are a helpful assistant."));

// Returns typed response!
AgentRunResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>(
    "Please provide information about John Smith...");

PersonInfo person = response.Result; // Strongly typed access

The Gap

The framework provides no way to combine these capabilities:

1. No RunAsync<T> extensions for AIAgent: The generic structured output methods only exist for ChatClientAgent
2. No memory support for ChatClientAgent: ChatClientAgentOptions has no AIContextProviderFactory property
3. Forced choice: Developers must choose between memory/tools OR structured output

Expected Behavior

Developers should be able to write:

// This should work but currently doesn't:
AIAgent agent = chatClient.CreateAIAgent(new ChatClientAgentOptions()
{
    Instructions = "You are a helpful assistant.",
    AIContextProviderFactory = ctx => new UserInfoMemory(...), // Memory
    ChatOptions = new() { Tools = [weatherTool] }              // Tools
});

// This should exist but currently doesn't:
AgentRunResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>(
    "Extract person information...", thread);  // Structured output

// Perfect: memory + tools + structured output in one agent!
PersonInfo person = response.Result;

Related Issues

• Issue Agent Abstraction Run Responses #86: Agent Abstraction Run Responses (discusses response structures)
• No existing issues found specifically addressing this API gap

Environment

• Microsoft Agent Framework: Latest (.NET)
• Platform: .NET 8+
• Package: Microsoft.Agents.AI

Repository Analysis

This issue is based on analysis of the official samples:

• Step03: Tools with AIAgent
• Step05: Structured Output with ChatClientAgent
• Step13: Memory with AIAgent

---

This limitation significantly impacts developer experience and prevents the framework from supporting common AI agent patterns. A unified interface would make the Microsoft Agent Framework more competitive with other agent frameworks that provide integrated memory, tools, and structured output capabilities.

[westey-m]

@napgab, thanks for the feedback.

Step 03 and Step 13 in the samples are actually also using ChatClientAgent, even though the variable is defined as AIAgent in the sample. Note that AIAgent is the base class and common interface for all agent types including ChatClientAgent.

The following also works:

ChatClientAgent agent = chatClient.CreateAIAgent(new ChatClientAgentOptions()
{
    Instructions = "You are a friendly assistant.",
    AIContextProviderFactory = ctx => new UserInfoMemory(
        chatClient.AsIChatClient(), 
        ctx.SerializedState, 
        ctx.JsonSerializerOptions)
});

The intention with the samples is to show that while we may be creating different agent types in different samples, they can all be used interchangeably once created via the base AIAgent type, hence the upcasting to AIAgent.

The structured output sample however is demonstrating a ChatClientAgent specific feature where a specific structured output type is requested at invocation time. This is not something that will necessarily be supported by all agent types. E.g. with a custom AIAgent implementation, the point of the agent may be in conflict with structured output. Consider such an implementation that is intended to produce html only. In this case asking for a specific json schema at runtime will not work. This sample therefore doesn't upcast to AIAgent so that it can demonstrate usage of RunAsync<T> that is only available on ChatClientAgent.

CC @SergeyMenshykh