Multi Agent Application - Managing State/Chat History

[hellodhana]

We are building Teams based Chat bot based on above architecture. We are making use of Azure Bot Services with Python based backend running in Azure App service. This App Service runs in a service plan which has multiple instances.

Our backend contains 4 ChatCompletion agents collaborating using AgentGroupChat.

Please clarify below.
a) Since our App Service is inherently stateless and a teams message can land on any of instances in service plan, we need to have a persistent store(like cosmos) to maintain and load conversation/chat history? Is this assumption right?
b) How do we instantiate AgentGroupChat with chat history loaded from persistent store? We tried passing chat history through chat_history while instantiatin AgentGroupChat instance. But Agents couldnt get the context from history.

[radrad]

A quick prompt for your question b. with ChatGPT o4-mini-high (with Search) generated this. See if it will work.

Here’s the pattern you should follow to “replay” a previously‐persisted chat into an AgentGroupChat so that all of your agents see the full history and can use it as context:

---

1. Load your persisted messages into ChatMessageContent

First, fetch your saved conversation from wherever you persisted it (database, file, Cosmos DB, etc.) and map each record into a ChatMessageContent:

// Suppose you have a POCO representing saved entries:
public class SavedMessage {
    public string Role { get; set; }      // "user", "assistant", "developer", etc.
    public string Content { get; set; }   // the message text
    public string? AuthorName { get; set; }// optional, for function-calling author
    public DateTime Timestamp { get; set; }
}

// Load from your store
IEnumerable<SavedMessage> persisted = await MyPersistence.LoadConversationAsync(conversationId);

// Transform into Semantic Kernel messages
var history = persisted
    .OrderBy(m => m.Timestamp)
    .Select(m => new ChatMessageContent(
          Enum.Parse<AuthorRole>(m.Role, true),
          m.Content,
          m.AuthorName))
    .ToList();

---

2. Instantiate your agents & AgentGroupChat before adding history

You cannot pass a pre-built history into the constructor; instead you create the group chat first, then replay the history:

// 1) Build your kernel and any plugins you need
var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(/*…*/)
    .AddAzureOpenAITextEmbeddingGeneration(/*…*/)
    .Build();

// 2) Instantiate your agents (ChatCompletionAgent, OpenAIAssistantAgent, AzureAIAgent, etc.)
var planner = new ChatCompletionAgent(/*…*/);
var searcher = new ChatCompletionAgent(/*…*/);
// …more agents…

// 3) Create the group chat, configuring your selection & termination strategies
var groupChat = new AgentGroupChat(planner, searcher /*, …*/)
{
    ExecutionSettings = new AgentGroupChatSettings
    {
        SelectionStrategy = new KernelFunctionSelectionStrategy( /*…*/ ),
        TerminationStrategy = new KernelFunctionTerminationStrategy( /*…*/ )
    }
};

---

3. Replay the persisted history

Now push each saved message into the chat’s history—this is exactly the same as if you had called AddChatMessage in real time:

// Replay all historical messages:
foreach (var msg in history)
{
    groupChat.AddChatMessage(msg);
}

At this point, all of your agents share the same chat history. You can even add a developer‐level system prompt here (by calling AddChatMessage(new ChatMessageContent(AuthorRole.Developer, "..."))) if you didn’t include it in persisted history.

---

4. Inject the new user turn and invoke

Finally, append the new user message and kick off the group chat:

// 4) Add the latest user request
groupChat.AddChatMessage(new ChatMessageContent(AuthorRole.User, userInput));

// 5) Run the multi-agent workflow
await foreach (var response in groupChat.InvokeStreamingAsync(cancellationToken))
{
    Console.Write(response.Content);
}

// 6) Mark complete (optional)
Console.WriteLine($"Done – chat complete? {groupChat.IsComplete}");

---

FAQ & Common Pitfalls

• Agents “don’t see” the old messages?
  Make sure you call AddChatMessage(...) before invoking. If you replay after calling InvokeAsync, they won’t factor in past turns.

• Order matters
  Always replay in chronological order—Semantic Kernel uses the sequence to build its prompt.

• Developer vs. System roles
  If you have a system or developer prompt, persist it as part of the history so it can be replayed alongside user/assistant turns.

• Memory vs. History
  This approach only replays chat history. If you’re also using a long-term memory store (e.g. vector memory for RAG), you’ll need to reload that separately via your memory plugin and pass it into the Kernel.

With this pattern—instantiating AgentGroupChat first, then replaying your persisted ChatMessageContent list—each agent will see every past turn as context and behave exactly as if you’d never stopped the conversation.

[moonbox3]

@radrad, thanks for taking a try at the answer. The original message is asking for help with SK Python, instead of SK .NET.

a) Yes
b) If you create an AgentGroupChat (which is a child of AgentChat), you can pass in the history to seed the conversation with some chat message content messages. Are you seeing an explicit error? If so, can you provide the stack trace?

[hidhana]

While calling chat.history.serialize() getting below error.

File "C:\Users\044642\VSCodeWorkspace\SemanticKernel\chunking.venv\Lib\site-packages\semantic_kernel\contents\chat_history.py", line 347, in serialize
raise ContentSerializationError(f"Unable to serialize ChatHistory to JSON: {e}") from e
semantic_kernel.exceptions.content_exceptions.ContentSerializationError: Unable to serialize ChatHistory to JSON: Unable to serialize unknown type: <class 'azure.ai.projects.models._models.RunStepCompletionUsage'>

[hidhana]

For now I am removing 'usage' key from ChatMessageContent metadata before serializing data. I can feed it back in AgentGroupChat instantiation. Thanks