llm-advisors

A small CLI and web app that lets you run a "board of LLMs as advisors" using:

• Codex (OpenAI CLI)
• Claude Code
• Antigravity CLI
• Ollama (local models)

It does not use any extra API keys – it shells out to the official CLIs, which authenticate using your existing subscriptions/accounts, and to Ollama for local models.

Vibe coded in an afternoon with codex.

The behaviour is inspired by Karpathy’s llm-council, but instead of going via OpenRouter/API keys, it coordinates multiple CLIs you already use.

1. What it does

Run a structured, multi-turn “council” of LLMs via the CLIs you already have. It shells out to Codex/Claude/Antigravity/Ollama and orchestrates up to 4 turns:

• Turn 1 - baseline: advisors answer freely; chair synthesises and produces a summary object.
• Turn 2 - divergence: advisors get roles (Explorer/Skeptic/etc.) and push new ideas; chair produces a task sheet.
• Turn 3 - task solving: advisors solve the task sheet; chair builds a convergence-prep summary.
• Turn 4 - convergence: advisors propose finals; chair produces the final answer.

Roles rotate after turn 1 to avoid stagnation. All prompts/results are logged per conversation; the web UI shows live status and stores artefacts for later review. Claude aliases, Codex/Antigravity defaults, and Ollama models on your machine are shown as model options in the web UI. For Google Gemini-family models, use Antigravity CLI via agy/<model>.

https://youtu.be/7Xft86ihGfs

---

2. Requirements

You’ll need:

• Python: 3.11 or newer
• A working install of the following CLIs (pick the ones you care about):
  • codex (OpenAI / ChatGPT CLI)
  • claude (Claude Code CLI)
  • agy (Antigravity CLI)
  • ollama (local LLM runtime)

Each CLI must already be:

• Installed on your $PATH
• Logged in / configured to use whatever account or plan you have
• Working from the shell by itself (e.g. codex "hello", claude -p "hello", agy -p "hello", ollama run llama3.2 "hello")

Google announced that Gemini CLI stopped serving requests for unpaid tier, Google AI Pro, and Ultra users on June 18, 2026, and moved those users to Antigravity CLI: https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/ This project supports the Google model path through the agy provider.

For Ollama, you’ll also need at least one model pulled, for example:

ollama pull llama3.2

3. Install (with venv)

Use a virtual environment so dependencies stay contained:

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

You can also invoke it from the repo root without installing via ./llm-advisors "<question>", but the editable install inside a venv keeps your global Python clean.

4. Usage

Ask the advisors a question:

llm-advisors "When should I use vector search vs full text search?"

Useful flags:

• --members: override which providers answer/review (defaults to config). Accepts codex claude agy ollama and provider model overrides like codex/gpt-5.5, claude/sonnet, agy/Gemini 3.5 Flash (Medium), and ollama/llama3.1:8b.
• --chair: choose who synthesises the final answer (defaults to config).
• --turns: run multiple council rounds (opinions -> reviews -> chair) and feed the last chair answer into the next turn.
• --show-intermediate: print stage 1 answers and stage 2 reviews for each turn.
• --show-turns: print a short summary per turn after the final answer.
• --max-parallel: global max concurrent provider calls (default 4).
• --no-thinking: prefer faster direct responses by lowering/disabling model thinking where the provider CLI supports it.
• --ollama-parallel-mode: sequential (default) | limited | parallel.
• --ollama-max-parallel: when mode=limited, how many Ollama calls to allow.
• --log-dir: where to write conversation artefacts (default conversations/).
• --log-disabled: skip writing artefacts.

Examples:

# Ask only Codex and Claude, use Claude as chair
llm-advisors --members codex claude \
    --chair claude "How do I debounce an async function?"

# Ask specific model variants, show intermediate output
llm-advisors --members codex/gpt-5.5 claude/sonnet "agy/Gemini 3.5 Flash (Low)" ollama/llama3.1:8b \
    --chair claude/opus --show-intermediate "How does a vector work?"

If a provider CLI fails, you’ll see a ProviderError with the CLI exit code and stderr - fix the underlying CLI and rerun, or disable that provider (see config below).

Multi-turn and artefacts

• --turns N runs up to 4 structured turns:
  • 1 turn: baseline synthesis
  • 2 turns: baseline → final convergence
  • 3 turns: baseline → divergence → final convergence
  • 4 turns: baseline → divergence → task solving → final convergence
• Post-baseline turns assign rotating roles (Explorer/Skeptic/etc.) to push divergence.
• Structured artefacts (summaries, task sheets, convergence prep) feed each turn.
• Every run gets a conversation ID. Artefacts live under conversations/<id>/ as meta.json plus turn-01.json, turn-02.json, etc. Disable with --log-disabled.

Web UI

A minimal local UI is available:

llm-advisors-web

# or

python -m llm_advisors_cli.web

It serves on http://127.0.0.1:8000/ and lets you start conversations, pick advisors/chair/turns, watch live per-member status (with a stop control), and browse/delete past runs from conversations/. Codex, Claude, Antigravity, and Ollama model variants are shown as provider/<model> options where they can be discovered locally or inferred from supported aliases/configuration.

Screenshots

Home / start page (question, advisors, turn slider):

Live status during a run (roles + per-member progress):

Conversation detail with final answer plus turn artefacts:

5. Configuration

Configuration is optional. Defaults live in code (codex/gpt-5.2, agy/Gemini 3.5 Flash (Medium), codex/gpt-5.4, ollama/gemma4:latest as members; codex/gpt-5.5 as chair). If present, config is read from:

~/.config/llm_advisors/config.toml

Structure:

[general]
# order matters for labelling A/B/C/… 
members = ["codex/gpt-5.2", "agy/Gemini 3.5 Flash (Medium)", "codex/gpt-5.4", "ollama/gemma4:latest"]

# who synthesises the final answer
chair = "codex/gpt-5.5"
max_parallel = 4
thinking_enabled = true

[providers.codex]
enabled = true                      # set false to skip entirely
command = "codex"                   # override the binary name/path
model = "gpt-5.5"                   # default Codex model
extra_args = ["--no-color"]         # appended before the prompt

[providers.claude]
enabled = true
model = "sonnet"                   # default model when using bare `claude`
extra_args = ["-p"]                 # defaults used if not provided

[providers.agy]
enabled = true
command = "agy"
model = "Gemini 3.5 Flash (Medium)" # default model when using bare `agy`
extra_args = ["--print-timeout", "5m"]

[providers.ollama]
model = "llama3.2"                  # default model when using bare `ollama`
extra_args = []                     # e.g. ["--timeout", "60"]

[parallelism.ollama]
mode = "sequential"                 # sequential | limited | parallel
max_parallel = 1                    # used when mode = limited

[logging]
enabled = true
base_dir = "conversations"

Runtime flags always win over config values. To use a specific Antigravity, Claude, or Ollama model once, pass --members "agy/Gemini 3.5 Flash (Low)" claude/sonnet ollama/llama3.1:8b (or set chair to agy/..., claude/..., or ollama/...). You can also point command to a mock script if you want to stub providers during development.