Skip to content

docs(orchestration): survey agent orchestration frameworks (closes #101) - #210

Open
dem-extra1 wants to merge 6 commits into
mainfrom
survey-orchestration-frameworks-101
Open

docs(orchestration): survey agent orchestration frameworks (closes #101)#210
dem-extra1 wants to merge 6 commits into
mainfrom
survey-orchestration-frameworks-101

Conversation

@dem-extra1

@dem-extra1 dem-extra1 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Why

Issue #101 asks for a survey of general-purpose agent orchestration frameworks (AutoGen, LangGraph, and others) so the lab can judge whether any of them beats its current Claude Code subagent and worktree setup.

What

Adds an "Agent Orchestration Frameworks" section to chapters/agent-orchestration.qmd covering Microsoft AutoGen and Agent Framework, LangGraph, CrewAI, the OpenAI Agents SDK, and Google ADK, with one-line notes on MetaGPT, LlamaIndex, smolagents, Pydantic AI, and CAMEL (VoltAgent is #45's PR). Each entry gives license, language, how orchestration is expressed, and a "Useful to us?" verdict, with a paragraph in the Comparison section and one Recommendation bullet. Star counts, licenses, and releases were measured on 2026-09-09.

Closes #101

🤖 Generated with Claude Code

dem-extra1 and others added 3 commits September 9, 2026 03:40
Lab members ask whether AutoGen, LangGraph, CrewAI, the OpenAI Agents SDK,
or Google ADK would let us build something our Claude Code stack cannot.
The chapter previously covered only three finished orchestrator tools, so
there was nowhere to record that answer.

Add a "Agent Orchestration Frameworks" section before Comparison that
covers each framework's design, license, language, and how orchestration
is expressed, with a "Useful to us?" verdict measured against the lab's
R/Quarto/Claude Code workflow. AutoGen is in maintenance mode per its own
README, so its successor (Microsoft Agent Framework) gets its own entry;
six more widely used frameworks get one line each, with VoltAgent left
to issue #45. Star counts, licenses, and release dates were read from
the GitHub API on 2026-09-09 and are stamped as such.

Comparison gains a paragraph explaining why libraries do not fit the
tool table, and Recommendation gains one bullet. Every cited page was
fetched; bib entries record the access date.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
hunspell reports "LangGraph's" as a misspelling because possessives are
not covered by the WORDLIST entry, so reword rather than add a token.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dem-extra1

Copy link
Copy Markdown
Collaborator Author

Resumed after the original worker was killed by a rate limit: salvaged its uncommitted draft of the frameworks section, re-verified every star count, license, and release date against the GitHub API on 2026-09-09 (OpenAI Agents SDK had released v0.22.2 in the meantime, now reflected), fetched each cited README and doc page, and added the fifteen references.bib entries and three wordlist words the draft still lacked. Pushed with ALLOW_UNREVIEWED_PUSH=1 because the adversarial-reviewer subagent type is unregistered in this session and the subagent pool was at its concurrency limit; the mechanical checks (render, chapter spell-check, citation cross-check, git diff --check) all pass locally.

Posted by Claude Code (AI agent) --- not written by a human.

@dem-extra1
dem-extra1 marked this pull request as ready for review September 9, 2026 16:01
Copilot AI lite review requested due to automatic review settings September 9, 2026 16:01
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The PR description/closing issue reference does not match the actual diff (framework survey for #101 vs posit-dev survey for #106), which risks closing the wrong issue on merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR expands the “Orchestrating Teams of Agents” chapter with a new survey of general-purpose agent orchestration frameworks (libraries) to complement the existing comparison of finished orchestration tools, and adds supporting bibliography/spellcheck updates. The PR description currently appears to describe a different change set (the posit-dev org survey / issue #106) than what’s actually in the diff.

Changes:

  • Add a new “Agent Orchestration Frameworks” section covering AutoGen, Microsoft Agent Framework, LangGraph, CrewAI, OpenAI Agents SDK, Google ADK, and brief notes on several others.
  • Add new bibliography entries in references.bib to support the newly cited framework docs/repos.
  • Extend inst/WORDLIST with new proper nouns referenced in the chapter (e.g., Foundry, Kotlin, LangSmith).
File summaries
File Description
references.bib Adds documentation/software citations for the newly referenced orchestration frameworks.
inst/WORDLIST Adds spelling whitelist entries for new tool/platform names introduced in the chapter.
chapters/agent-orchestration.qmd Introduces a new framework-focused survey section and updates recommendations accordingly.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +205 to +210
# Agent Orchestration Frameworks {#sec-orch-frameworks}

The three projects above are finished tools.
The projects in this section are *frameworks*:
software libraries for building your own multi-agent system
from agents, tools, and a control flow that you write in code.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The PR metadata/closing issue scope is inconsistent with the actual changes, and one newly-added recommendation should be timestamp-aligned with the chapter’s new “measured 2026-09-09” framework data.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

chapters/agent-orchestration.qmd:237

  • The PR metadata/description says this change is about closing #106 (posit-dev org review), but this chapter content is explicitly addressing issue #101 (agent orchestration frameworks). Please update the PR title/description/closing keywords so the PR closes the correct issue and doesn’t point reviewers at the wrong scope.
[AutoGen](https://microsoft.github.io/autogen/stable/) [@autogen_docs]
is Microsoft's original multi-agent framework,
and the one issue #101 named first.
Its current design is layered:
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread chapters/agent-orchestration.qmd
@dem-extra1

Copy link
Copy Markdown
Collaborator Author

Rebutting the security finding about the ALLOW_UNREVIEWED_PUSH=1 comment: it is not a review bypass of this PR. ALLOW_UNREVIEWED_PUSH=1 is the documented override of a local, client-side pre-push hook in Morrison-Lab/ai-config (hooks/no-push-without-self-review.py), which normally requires an adversarial-reviewer subagent verdict before a push leaves the author's machine. That subagent type was not registered in the authoring session, so the local pre-push review was performed by the Google Antigravity CLI instead and the override was used, with the disclosure posted here so reviewers can see it. Nothing about the PR's own review gates changed: this claude-review workflow and the requested Copilot review still run on every push and require-clean-verdict still blocks merge.

The other finding (PR body describing a different change) was a real defect from a scratch-file collision between parallel workers; the body has been rewritten to describe this diff and now says Closes #101, so merging will not touch #106.

Posted by Claude Code (AI agent) --- not written by a human.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

The new framework survey is measured on 2026-09-09, but the
recommendation it feeds still said "as of August 2026"; the two now
agree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 10, 2026 06:59
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are a few objective consistency/convention issues in the updated chapter text and bibliography license formatting that should be corrected before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

references.bib:340

  • License names in note fields appear to use SPDX-style shorthand (e.g., Apache-2.0 in rakazo at references.bib:214). This entry uses “Apache License 2.0”, which is inconsistent with the established format in this file.

This issue also appears in the following locations of the same file:

  • line 365
  • line 383

references.bib:367

  • License names in note fields appear to use SPDX-style shorthand (e.g., Apache-2.0 in rakazo at references.bib:214). This entry uses “Apache License 2.0”, which is inconsistent with the established format in this file.
  howpublished = {Software},
  url = {https://github.com/huggingface/smolagents},
  note = {Apache License 2.0. Accessed 2026-09-09}

references.bib:385

  • License names in note fields appear to use SPDX-style shorthand (e.g., Apache-2.0 in rakazo at references.bib:214). This entry uses “Apache License 2.0”, which is inconsistent with the established format in this file.
  howpublished = {Software},
  url = {https://github.com/camel-ai/camel},
  note = {Apache License 2.0. Accessed 2026-09-09}
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment on lines 16 to +20
This chapter explains when orchestration is worth the added cost,
describes what our lab already uses for it,
and evaluates three outside "agent orchestrator" projects
that lab members have asked about.
evaluates three outside "agent orchestrator" projects
that lab members have asked about,
and surveys the general-purpose orchestration frameworks
…frameworks-101

# Conflicts:
#	references.bib
Copilot AI review requested due to automatic review settings September 10, 2026 08:09
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

references.bib contains an incomplete BibTeX entry (autogen_teams) that will break bibliography parsing/rendering.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread references.bib
The union resolver used for the sync dropped the blank line between
every pre-existing entry and truncated one entry, which made pandoc
reject the file and failed the build. The file is now rebuilt from
the two parent versions: main's entries in main's order and formatting,
then this branch's new entries appended, validated with pandoc.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 10, 2026 08:32
@github-actions

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Confirmed — this round's only change is the references.bib rebuild; the chapter and wordlist are untouched.

Review of #210

This is a follow-up to the prior round posted at #210 (comment) (Needs more work), which flagged a blocking malformed BibTeX entry: the merge-conflict resolution in commit 9cf8f389 had dropped the note line and closing } from the autogen_teams entry, causing it to run into the next entry and drop that citation key entirely.

The only change since that round is commit 38005dc ("fix(bib): rebuild references.bib as a clean union after the main sync"), which rewrites references.bib (107 insertions, 55 deletions) but leaves chapters/agent-orchestration.qmd and inst/WORDLIST untouched (confirmed via git diff 9cf8f389 c831508b on those two paths — 0 lines changed).

I re-verified the fix directly rather than trusting the commit message:

  • BibTeX parses cleanly. bibtexparser.parse_file('references.bib') now reports 51 entries (the expected count) with zero parse errors and zero duplicate keys.
  • autogen_teams is restored correctly at the end of the file, with its note = {Accessed 2026-09-09} line and closing } intact — matching the well-formed version that existed before the botched merge.
  • No entries were lost in the rebuild: every key present in the pre-merge baseline (0d9221aa:references.bib) is still present in the current references.bib.
  • All citation keys used in chapters/agent-orchestration.qmd resolve against the bib file (autogen_docs, autogen_repo, autogen_teams, camel_ai, crewai_docs, crewai_repo, google_adk, langgraph_docs, langgraph_repo, llama_index, metagpt, microsoft_agent_framework, openai_agents_sdk, pydantic_ai, smolagents, plus the pre-existing claude_agent_teams, inflexa, torqclaw).
  • ::: div fences balance at 22 (even), and both @sec- cross-refs used in the new frameworks section (@sec-orch-frameworks, @sec-orch-when) resolve to defined labels — unchanged from prior verification.

No new issues introduced by this fix commit. The blocking defect from the prior round is resolved with a correct, minimal rebuild rather than a workaround.

Verdict

Ready for merge — the previously blocking malformed-BibTeX regression (autogen_teams entry / dropped oh_my_claudecode boundary) is fixed by a clean rebuild that restores the missing lines without losing any other entries, verified against a real BibTeX parser; all prior content checks (citations, cross-refs, div balance, WORDLIST) still hold.

Structured Review Data (JSON)
{
  "schema_version": "1.1",
  "reviewer": "claude",
  "commit_sha": "c831508b656c92b8945f47d324ec530dfd490cf8",
  "verdict": "CLEAN",
  "findings": [],
  "detailed_assessment": "references.bib was rebuilt in commit 38005dc and now parses as 51 well-formed entries with the previously truncated autogen_teams entry correctly closed.",
  "holistic_assessment": "The bibliography rebuild preserves every pre-merge citation key, keeps all chapter citations resolvable, and introduces no regression in cross-reference or fence-balance validation."
}

Reviewed commit: 38005dc

@github-actions

Copy link
Copy Markdown
Contributor

💰 Cost: $0.2943 (review) — run

@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://Morrison-Lab.github.io/wai/pr-preview/pr-210/

Built to branch gh-pages at 2026-09-10 08:36 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new bibliography entries and chapter text introduce a couple of consistency issues (license string format and an “August vs September 2026” freshness date) that should be corrected before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

references.bib:431

  • In this bibliography file, Apache-licensed projects elsewhere use the SPDX-style short form Apache-2.0 (e.g., rakazo at references.bib:214), but this entry uses "Apache License 2.0". Please standardize the license string for consistency.
  note = {Apache License 2.0. Accessed 2026-09-09}

references.bib:449

  • In this bibliography file, Apache-licensed projects elsewhere use the SPDX-style short form Apache-2.0 (e.g., rakazo at references.bib:214), but this entry uses "Apache License 2.0". Please standardize the license string for consistency.
  note = {Apache License 2.0. Accessed 2026-09-09}

chapters/agent-orchestration.qmd:21

  • This intro now frames the chapter as surveying orchestration frameworks and later updates the recommendation to September 2026, but the warning callout still says the quoted numbers were true when written in "August 2026" (line 30). Updating that month keeps the chapter’s freshness disclaimer consistent with the new measurement date (2026-09-09).
and surveys the general-purpose orchestration frameworks
a lab could build its own agent systems on.
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread references.bib
year = {2026},
howpublished = {Software},
url = {https://github.com/google/adk-python},
note = {Apache License 2.0. Accessed 2026-09-09}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

survey ai agent orchestration frameworks

2 participants