Skip to content

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Hooded jurors seated in a circle around a brazier inside a night-time temple, a robed judge elevated behind the fire

jury

Wisdom of crowds for AI agents — zero code, one document.


When an AI agent answers a hard judgment question in a single pass, you get one perspective, one blind spot, and full confidence. jury replaces that with a panel: several workers with deliberately different lenses reason independently, a stronger judge aggregates them by counting reasons rather than heads, and what the panel cannot settle gets escalated instead of guessed. The output is never a decision — it is a decision draft with rationale that a human can approve in five minutes.

There is nothing to install. The entire system is one vendor-neutral document, PROTOCOL.md, that any agent CLI — Claude Code, Codex, Gemini CLI, anything that reads an instruction file — executes by following it.

How it works

Step What happens
0 · Classify Execution work (search, mechanical edits, checks) skips the jury — plain fan-out. Judgment work — where reasonable agents could disagree and being wrong is expensive — gets a panel.
1 · Dispatch N workers in parallel, each with a different lens (correctness, security, cost…), each blind to the others. Same prompt N times is resampling, not diversity: the copies share one blind spot.
2 · Aggregate A judge stronger than the jurors collects structured positions (claim, evidence, facts assumed, trade-offs, confidence) and resolves splits by checking assumed facts — not by majority.
3 · Synthesize A decision draft: the recommendation, the strongest dissent (never laundered away), the facts everything hinges on, and what was escalated. The human decides.

Three things escalate past the jury: a vote that stays split after fact-checking, items that are inherently deep judgment (architecture, "don't build it" calls), and — the counterintuitive one — unanimity on a hard question. On genuinely difficult judgments, 5/5 agreement is not confidence; it is the signature of a shared blind spot. Healthy juries split, and a strong judge picks the side with the evidence.

Why it works (and where it stops working)

Condorcet's jury theorem: a crowd beats its members only while errors are independent and each member is better than chance. Below that competence threshold the theorem inverts — adding jurors makes the crowd converge on a confident wrong answer. So jury spends its structure on keeping errors independent (lenses, parallel dispatch, no chaining) and refuses to let the crowd vote on questions where individual competence is the binding constraint. Those go up a model tier, not out to more workers.

The practical boundary, from the coverage analysis in the appendix: exploration, mechanical edits, and candidate-finding delegate beautifully (~70–95%); final bug confirmation, architecture calls, and plan coherence do not (~10–40%) — no amount of parallelism substitutes for one strong mind on those.

Use it

Point your CLI's instruction file at the protocol — this repo's own CLAUDE.md / AGENTS.md / GEMINI.md are exactly that, three thin pointers at one canonical text. Then ask your agent a judgment question and watch it empanel.

In its first field run, a three-lens panel split 2:1 on a repository-layout question; the dissenting juror's blocking assumption failed a fact-check the risk-lens juror had already run — independently, without either seeing the other. The panel converged on evidence, the human approved the draft, and the "facts assumed" field did precisely the job it was designed for.

Files

  • PROTOCOL.md — the canonical text. Everything else defers to it.
  • CLAUDE.md, AGENTS.md, GEMINI.md — per-CLI pointers, three lines each.
  • banner.svg — the panel in session.

About

wisdom of crowd for ai agents

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors