How to stream GroupChat response?

[Laxmi530]

Hi,
I am starting this discussion as I have some questions regarding GroupChat:

1. How can I stream the final response from GroupChat? I am using a Streamlit UI and would like to stream the output there.
2. Suppose a general query is received that can be answered by the LLM without using any tools. In that case, which GroupChat should be used, and how should it be configured? Is there a specific parameter that needs to be set, or what is the recommended approach?
3. In GroupChat, is there an option to share the chat history with the model? If so, how many past conversations are shared, and is it possible to control the number of conversations being passed to the model?
4. Can we cache the final response from the GroupChat? If so, how many conversations can be cached?

Looking forward to your guidance and suggestions.
Thank you!

[SongChiYoung]

1. GroupChat Message Streaming (for Streamlit)

AutoGen provides run() and run_stream() methods for GroupChat execution.

• run_stream() is an async generator that yields responses after each Agent completes its turn.
  • Note: This is not token-level streaming; each chunk is the full response from one Agent.
  • It's useful if you want to see the internal turn-by-turn conversation between Agents.
• run() returns the final result after the entire GroupChat interaction is complete.
  • If you only need the final response in your Streamlit UI, using run() is simpler and more efficient.

📎 Code Example:

• If you'd like to see a code example using GroupChat, you can refer to this repository: link
• Additionally, if you're looking for Streamlit + OpenAI SDK integration, I have a repository where AutoGen is wrapped as an OpenAI API-compatible FastAPI server for easier usage: link

---

2. Choosing the Right GroupChat Type

AutoGen offers different GroupChat types depending on how Agents are selected:

• RoundRobinGroupChat: Agents are called in a fixed, sequential order.
  • Simple, predictable workflow.
• SelectorGroupChat: Agent selection is decided dynamically by the LLM.
  • Flexible, adaptive flow based on context.
• Here is official document for you link

📌 Tip:

• GroupChat selection is independent of Tool usage, but SelectorGroupChat works better with diverse Agent setups.
• For beginners, RoundRobinGroupChat is a good starting point to understand the structure.
• Here is official document for you link

---

3. Message Sharing & Retention Policy (model_context)

By default, all messages are shared with all Agents in AutoGen.
Each Agent manages these messages through model_context:

Here is official document for you link

• UnboundedChatCompletionContext: All messages are retained indefinitely (default).
• BufferedChatCompletionContext: Only the last N messages are kept.
• and more contexts can be found in the documentation.

🛠️ Customization:

• You can define custom contexts to filter or summarize messages, or selectively share them.
• My repository offers extended model_context options, including:
  • Summarization-based retention
  • Conditional modification trigger logic
• If you have ideas or need a specific context behavior, feel free to open an Issue in my repo. Depending on priority, it might be added as a built-in feature.
  link

---

4. GroupChat Caching

AutoGen does provide caching functionality, but I have not used it directly.

• Caching exists internally, but I cannot provide detailed explanations on how it works.
• However, there are some discussions around caching usage in the AutoGen community which you may find helpful: link

---

[SongChiYoung]

Here is other streamlit example for you
https://github.com/microsoft/autogen/blob/main/python/samples/agentchat_streamlit/main.py

you could find examples in the link
https://github.com/microsoft/autogen/blob/main/python/samples

[Laxmi530]

Hi @SongChiYoung,
Thank you for your detailed explanation—I truly appreciate the insights you’ve provided.

1. I attempted to use the run() method to enable streaming in the Streamlit UI. While the first query works as expected, I encounter a RuntimeError on the second query, as shown in the attached screenshot. I've tried several approaches to resolve it, but the issue persists. If you have any suggestions or guidance on how to address this error, I’d be grateful.

2. I’ve successfully implemented RoundRobinGroupChat—thank you for the documentation; it was very helpful. I’m currently working on SelectorGroupChat. I have multiple agents configured: one is GeneralAgent (without any tools) and another is WebSearchAgent (with web search capabilities). In some cases, a query that can be answered using LLM knowledge alone is still being routed to WebSearchAgent or other agents, instead of GeneralAgent. I’ve included a screenshot for reference. Is there a way to control this behavior—perhaps through prompt engineering? Specifically, should I be modifying the selector_prompt or the system_message of GeneralAgent?

3. I’ll look into the Message Sharing & Retention Policy further—thank you again for the helpful explanation and links.
4. As for GroupChat caching, that’s on my to-do list to implement next.

Thanks again for your support.

[SongChiYoung]

1. In my experience, I encountered a similar error when a previous message still contained the word “TERMINATE.”
   If you are using TextMentionTermination("TERMINATE"), but a prior message already includes “TERMINATE,” the team may immediately terminate as soon as it starts.
   To prevent this, you need to remove or filter out any previous “TERMINATE” messages before sending the next message to the team.

2. This is a tricky issue, haha.
   In my opinion, the most practical solution is to adjust the prompt engineering — specifically, we can modify the system prompt of the SelectorGroupChat to guide agent selection more precisely.
   Internally, when SelectorGroupChat selects the next agent, it refers to each agent’s description.
   By default, the descriptions are automatically inserted into the SelectorGroupChat’s system prompt.

So, first, you should customize the description field for each agent carefully.
Then, you should modify the system prompt of the SelectorGroupChat itself.
(You should also review the default system prompt of SelectorGroupChat to understand how it combines agent descriptions and names.)