Step 7: replace hosted handoff with a manager-led group chat#20
Open
pmalarme wants to merge 5 commits into
Open
Step 7: replace hosted handoff with a manager-led group chat#20pmalarme wants to merge 5 commits into
pmalarme wants to merge 5 commits into
Conversation
…g) before group chat rewrite Snapshots the HandoffBuilder-based Step 7 with the investigated termination_condition + silent-routing workaround for the hosted multi-turn crash (Unexpected content type while awaiting request info responses). Archived for future investigation in issue #19; Step 7 is moving to a group chat orchestration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace the HandoffBuilder-based Step 7 with a GroupChatBuilder group chat where the Coordinator is the manager (orchestrator_agent) that selects the next specialist each round and terminates with the final answer. The hosted handoff pattern crashed on follow-up questions (Unexpected content type while awaiting request info responses) because HandoffBuilder parks in IDLE_WITH_PENDING_REQUESTS awaiting a function_result, while ResponsesHostServer delivers follow-ups as plain text. Group chat never parks: the manager terminates each turn cleanly, so follow-ups just continue the conversation. Archived handoff code and investigation in issue #19 (commit a01ae6d). Keeps docs, solutions, and step_files in sync. Skills ride on the Activities specialist since the manager returns structured output and cannot carry tools/context providers. Lint passes with 0 failures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Update the solution README.md and main.py comment (stale HandoffBuilder wording -> group chat), which the initial rewrite missed. - Align each specialist description= in the solution and starter example to the exact agents/*/agent.yaml description: text, so 'copy from the slice' yields the reference solution. - Raise max_rounds 20 -> 40 and document that the round counter is checkpoint-restored, so the cap is cumulative across the whole hosted conversation (not per turn); add a caveat that a very long conversation can exhaust it. - Reword the model tip: group chat cleans tool-call content before broadcast, so it replays each specialist's tool-grounded response text, not raw tool output. Issue #19 updated to note the inline snippet is simplified and to point at the a01ae6d permalink (with termination_condition + require_per_service_call_history_persistence) for reviving the handoff. Lint 0 failures; both coordinator files compile; step 7 renders. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates Workshop Step 7 to avoid the hosted multi-turn instability of HandoffBuilder by pivoting the multi-agent orchestration to a manager-led GroupChatBuilder shape, while keeping the “single hosted agent via workflow.as_agent()” deployment model intact.
Changes:
- Replaces the Step 7 runtime handoff graph with a
GroupChatBuildergroup chat where the Coordinator acts as the orchestrator/manager and specialists are participants. - Updates specialist slices/instructions to match the new “manager selects next speaker; specialists report and stop” flow, including routing via
description=. - Updates Step 7 workshop docs + reference solution + step scaffolds + manifest metadata to reflect the group chat model.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| .workshop/step_files/07/coordinator.py | Updates the learner scaffold to build a GroupChatBuilder group chat and explains manager/participant responsibilities. |
| .workshop/step_files/07/agents/flights/agent.yaml | Adjusts specialist boundary text to “report findings and stop” for group chat. |
| .workshop/step_files/07/agents/hotels/agent.yaml | Adjusts specialist boundary text to “report findings and stop” for group chat. |
| .workshop/step_files/07/agents/activities/agent.yaml | Updates Activities guidance for being selected last and producing the guarded PDF deliverable in a group chat flow. |
| .workshop/solutions/07-multi-agent/travel_assistant/requirements.txt | Updates comments to reflect GroupChatBuilder as the multi-agent orchestration. |
| .workshop/solutions/07-multi-agent/travel_assistant/main.py | Updates Step 7 solution commentary to refer to group chat instead of handoff. |
| .workshop/solutions/07-multi-agent/travel_assistant/coordinator.py | Rewrites the Step 7 reference solution to use GroupChatBuilder with the Coordinator as orchestrator_agent. |
| .workshop/solutions/07-multi-agent/travel_assistant/agents/flights/agent.yaml | Keeps solution slices aligned with the new group chat specialist behavior. |
| .workshop/solutions/07-multi-agent/travel_assistant/agents/hotels/agent.yaml | Keeps solution slices aligned with the new group chat specialist behavior. |
| .workshop/solutions/07-multi-agent/travel_assistant/agents/activities/agent.yaml | Keeps solution slices aligned with the new group chat specialist behavior and deliverable ownership. |
| .workshop/solutions/07-multi-agent/travel_assistant/agent.manifest.yaml | Switches manifest metadata from handoff to group_chat { manager, specialists }. |
| .workshop/solutions/07-multi-agent/README.md | Updates the Step 7 solution README to describe the manager-led group chat architecture. |
| .workshop/docs/steps/07-multi-agent.md | Rewrites the Step 7 doc to teach group chat orchestration and documents the hosted follow-up crash rationale. |
| > **The Foundry skill is optional here.** The **Activities** specialist owns the two skills that shape the final answer — the LOCAL `travel-guide` skill (renders the shareable PDF trip guide) and the Foundry `response-guardrails` skill carried from Step 6 — and the checked-in **solution is the Foundry-enabled reference**: its `coordinator.py`, `ACTIVITIES_INSTRUCTIONS`, and `.env.example` all wire the guardrails skill in and treat it as required. If you built it in Step 6, keep it: serve both skills from your skills provider and keep the "always apply `response-guardrails`" line in `ACTIVITIES_INSTRUCTIONS`. If you **skipped** the Foundry skill (for example your Foundry project can't allow public network access — see Step 6), leave `FOUNDRY_SKILL_NAMES` unset, drop the `response-guardrails` line from `ACTIVITIES_INSTRUCTIONS`, and serve only the local `travel-guide` skill — carry your Step 6 *local-only* skills provider rather than the solution's Foundry-enabled `_build_skills_provider`. The local `travel-guide` skill still renders the PDF and nothing else in this step depends on the Foundry skill. In practice, if you couldn't build the Foundry skill in Step 6 you already made your skills provider treat it as optional there — just carry that same local-only provider forward; there's nothing extra to redo here. | ||
|
|
||
| > **Why the skills ride on Activities, not the Coordinator.** A skills provider is a **context provider that registers its skill *tools*** on whichever agent holds it. In a runtime handoff the Coordinator is the only participant invoked **twice** — once to route, then again to synthesize after a specialist hands back — and with `default_options={"store": False}` the framework replays the whole tool-call history on that second call. Attaching a **tool-producing** context provider (the skills provider) to the Coordinator desyncs that replay and the service rejects it (`400 No tool call found for function call output`). Leaf specialists are invoked **once**, so they carry context providers safely — which is why the skills live on the Activities leaf here. The trade-off: only Activities' output is guarded, not the Coordinator's final synthesis, and the Coordinator has to *route* the deliverable through Activities rather than being *structurally* forced to. That's a real limitation of a pure router, and it's exactly what **Step 8** fixes — its workflow adds a dedicated `finalize` node that owns the deliverable and guards the actual final answer. | ||
| > **Why the skills ride on Activities, not the Coordinator.** A skills provider is a **context provider that registers its skill *tools*** on whichever agent holds it. But the Coordinator here is the group chat **manager**: each round it returns a *structured* routing decision (which specialist speaks next, or terminate with the final answer), so it can't also carry tools or a tool-producing context provider like the skills provider. The skills therefore live on the **Activities specialist** — a normal participant that runs its tools and skills freely. The trade-off: only Activities' output is guarded, not the manager's final synthesis, and the manager has to *route* the deliverable through Activities rather than being *structurally* forced to. That's a real limitation of a manager-plus-specialists chat, and it's exactly what **Step 8** fixes — its workflow adds a dedicated `finalize` node that owns the deliverable and guards the actual final answer. |
The Step 7 rewrite from HandoffBuilder to GroupChatBuilder changes how multi-agent routing interacts with RAG. In a group chat every message is broadcast to every participant and the manager only selects the next speaker; it never hands a reformulated single-topic sub-request. So a grounded specialist still builds its search query from the full shared transcript — routing alone does not narrow retrieval. Reword option 2 of the 'grounding several destinations' note to say so and point at options 1 and 4 for the actual narrowing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Avoid overclaiming that a specialist grounds on the whole shared transcript — the AgentExecutor clears its message cache after each run (_agent_executor.py), so the exact accumulation is more nuanced. Scope the 'manager hands no sub-request' claim to Step 7's default manager-led group chat, since context_mode/context_filter and additional_instruction allow custom narrowing. The actionable point stands: routing alone does not reliably narrow retrieval; pair it with option 1 or 4. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Step 7 built the multi-agent TravelBuddy with
HandoffBuilderexposed as a single hosted agent. In practice, handoff + hosted agents is unreliable: the first question works, but any follow-up in the same conversation crashes with:{ "error": { "code": "server_error", "message": "Unexpected content type while awaiting request info responses." } }Root cause (verified against agent-framework source):
HandoffBuilderis human-in-the-loop by default — after each non-handoff turn it callsctx.request_info(...)and parks the workflow inIDLE_WITH_PENDING_REQUESTS, expecting the next input as afunction_result. But the hostedResponsesHostServerdelivers follow-ups as plain text, soWorkflowAgent._extract_function_responses()raises. Atermination_conditionworkaround stopped the crash but regressed multi-hop routing (fragile, model-dependent silent-routing requirement).Full investigation, symptoms, and root cause are archived in #19, pinned to commit
a01ae6d(with permalinks to the exact handoff code) so the handoff path can be revived later.What changed
Pivots Step 7 to a manager-led
GroupChatBuildergroup chat, where the Coordinator is the group chat manager (orchestrator_agent):GroupChatBuilderhas zerorequest_infoin its default flow, so each turn completes IDLE — a follow-up question just continues the conversation. The follow-up crash is structurally gone.travel-guidePDF +response-guardrailsskills therefore ride on the Activities specialist, which the manager selects last.description=(feeds the manager's auto-generated routing prompt), slicedtools, and slicedcontext_providers(RAG for Hotels/Activities, skills for Activities).store=Falseis kept on every participant — safe here because group chat cleans tool-call content before broadcasting.handoff:→group_chat: { manager, specialists }; docs, solution, and starter all rewritten in sync.Notable design detail
max_roundsis cumulative across turns (the round counter is checkpoint-restored, not reset per user turn). Set to40for headroom across a multi-turn planning session, with a doc/comment caveat that a very long conversation could eventually exhaust it.Verification
python .workshop/scripts/lint_steps.py→ 0 failures (2 pre-existing unrelated warnings).coordinator.pyfiles compile; step 7 renders cleanly.Not yet run: a live hosted multi-turn run against Azure (
azd ai agent run+ follow-up). The fix is verified structurally via agent-framework source inspection.Archives #19.