Skip to content

vu1n/canopy

BranchesTags

Repository files navigation

Canopy

Token-efficient codebase indexing and querying for LLM agents.

Quick Start with Claude Code

1. Install

curl -fsSL https://raw.githubusercontent.com/vu1n/canopy/main/install.sh | sh

This installs canopy + canopy-mcp to ~/.local/bin/ and configures Claude Code automatically.

Options:

  • --no-claude-setup: skip Claude Code MCP configuration.
  • --prefix /usr/local: install to a custom location.
Build from source 
git clone https://github.com/vu1n/canopy.git
cd canopy
make install        # builds + installs canopy + canopy-mcp
make setup-claude   # configures Claude Code

2. Use It

Claude Code now has access to canopy tools. Ask questions like:

  • "Find the AuthController class" -> canopy_query(symbol="AuthController")
  • "How does authentication work?" -> canopy_query(pattern="auth")
  • "List all API endpoints" -> canopy_query(symbol="Controller") plus selective expands

Typical agent flow:

  1. Index relevant paths based on query intent (predictive lazy indexing).
  2. Return compact handles + previews instead of full files.
  3. Expand only handles needed for the final answer.

When to Use Canopy

Best For

Scenario Why
Large codebases (>1000 files) Predictive indexing avoids blocking on full upfront index
Symbol discovery Finds function/class definitions with file:line anchors
Cross-file tracing Follows execution paths across files
Subsystem analysis Scopes indexing/query to relevant areas
Parallel agents Shared index behavior is better for concurrent exploration

Skip Canopy For

Scenario Use Instead
Known file path Read tool directly
Literal text pattern Grep tool
File by name Glob / fd
Small repos (<500 files) Native tools are usually enough

How Canopy Works

Canopy treats repository understanding as a budgeted retrieval loop: broad cheap retrieval first, then selective expansion under explicit token and turn constraints.

Token-Efficiency Methods

  1. Handle-first retrieval canopy_query returns compact handles and previews first; full content is fetched with canopy_expand.
  2. Predictive lazy indexing Query intent predicts likely globs so indexing is targeted before search.
  3. Feedback-reranked retrieval Query/expand feedback in .canopy/feedback.db reranks future retrieval:
    • glob ranking (glob_hit_rate_at_k)
    • node-type priors (handle_expand_accept_rate)
  4. Retrieve -> local overlay -> merge (service mode) Service results are merged with local dirty-file overlays to keep answers fresh without full reindex.
  5. Guidance-driven evidence packs canopy_evidence_pack returns ranked handles plus guidance signals to stop querying and switch to expansion/synthesis.
  6. Low-confidence recursive planning (service mode) Service-side planning is enabled by default only when evidence confidence is low; otherwise it stays single-step.
  7. Expand churn suppression Recently expanded handles are de-prioritized in future expand_suggestion lists to avoid repeated expansions.
  8. Worst-case budget policy Retrieval is constrained by expand_budget, limit, and turn budget (MAX_TURNS in swarm tests).

Theory: Budgeted Retrieval Control Loop

Informal objective:

  • maximize answer utility (grounding, coverage, correctness)
  • minimize token cost and tail-risk of runaway context growth

Mental model for ranking:

score(handle) =
  w_text * lexical_relevance +
  w_type * node_type_prior +
  w_feedback * historical_expand_acceptance

The ranker optimizes expected utility per token, not just raw relevance.

Diagram: Retrieval and Ranking Loop

User Question
    |
    v
Predictive Scope Selection
  (keywords -> likely globs / symbols)
    |
    v
Initial Query (cheap)
  -> candidate handles + previews + token estimates
    |
    v
Handle Ranking
  (text relevance + type priors + feedback priors)
    |
    v
Budget Gate
  - expand top-k within budget
  - keep strict limit / expand_budget
    |
    +--> unresolved + budget left? -- yes --> re-query / re-rank / expand
    |                                       (iterate)
    |
    no
    |
    v
Synthesize Answer
  (grounded in expanded evidence)

Diagram: Service Mode Merge Loop

                    +------------------------------+
                    |        canopy-service        |
                    |  pre-indexed repo snapshots  |
                    +---------------+--------------+
                                    |
                                    v
User Question -> canopy-mcp -> Service Query/Expand (generation-tagged handles)
                                    |
                                    v
                          Service Candidate Set
                                    |
                                    v
                   Dirty-File Detector (local working tree)
                                    |
                    +---------------+---------------+
                    |                               |
              clean |                               | dirty
                    v                               v
            use service result               local incremental index
               directly                      on dirty subset only
                    |                               |
                    +---------------+---------------+
                                    |
                                    v
                      Local/Service Result Merge
                      (dedupe + freshness preference)
                                    |
                                    v
                          Grounded Final Answer

Diagram: Feedback Learning Loop

Query Issued
    |
    v
Handles Returned
    |
    v
Which handles were expanded?
    |
    v
Write events to .canopy/feedback.db
  - query_events
  - query_handles
  - expand_events
    |
    v
Compute derived signals
  - glob_hit_rate_at_k
  - handle_expand_accept_rate
  - node-type priors
    |
    v
Apply priors during future ranking
    |
    v
Better next-query ordering under same token budget

Token Economy (Illustrative)

Traditional approach:

Agent reads file1.ts (500 tokens)
Agent reads file2.ts (800 tokens)
Agent reads file3.ts (600 tokens)
Total: 1900 tokens

Canopy approach:

Agent queries "auth" -> 10 handles with previews (200 tokens)
Agent expands 2 relevant handles (400 tokens)
Total: 600 tokens

Use canopy feedback-stats to inspect local retrieval feedback metrics.

MCP Tools

Once configured, Claude Code has these tools:

canopy_query

Search indexed content. Returns handles with previews and token counts.

canopy_query(pattern="authentication")
canopy_query(symbol="AuthController", glob="src/**/*.ts")
canopy_query(patterns=["TODO", "FIXME"], match="any")
Param Type Description
pattern string Text pattern to search
patterns array Multiple patterns
symbol string Code symbol (function, class, struct, method)
section string Markdown section heading
glob string Filter by file glob
match any | all Multi-pattern mode
limit integer Max results (default: 16)
expand_budget integer Deprecated auto-expand toggle (default: 0, disabled)

canopy_evidence_pack

Build a compact ranked evidence set (no snippets) to minimize context bloat before expanding.

canopy_evidence_pack(pattern="authentication", max_handles=8, max_per_file=2)

Response includes guidance.stop_querying, guidance.recommended_action, and guidance.next_step so agents can transition from retrieval to synthesis without custom prompt rules.

canopy_expand

Expand handles to full content.

canopy_expand(handle_ids=["h1a2b3c...", "h5d6e7f..."])

canopy_status

Get index statistics.

canopy_invalidate

Force reindex of files.

canopy_agent_readme

Return usage guidance for agents/tool callers.

This guidance is canopy-first: prefer canopy retrieval over ad-hoc find/grep/rg for discovery, then expand selectively for synthesis.

CLI Usage

# Initialize canopy in a repository
canopy init

# Index files (MCP server auto-indexes on query; CLI requires explicit index)
canopy index

# Query the codebase
canopy query --pattern "authentication"
canopy query --symbol "AuthController"

# Expand handles to full content
canopy expand <handle_id>

# Check index status
canopy status

# Local feedback metrics
canopy feedback-stats

Operating Modes

Canopy supports two modes through the shared canopy-client runtime.

Standalone Mode

Default when no service URL is configured.

canopy query --pattern "auth"

Best for: solo developers, small/medium repos, quick setup.

Service Mode

For multi-agent and team workflows with shared indexing.

# Install and start service
curl -fsSL https://raw.githubusercontent.com/vu1n/canopy/main/install-service.sh | sh
canopy-service --port 3000

# Register and reindex repo
curl -X POST localhost:3000/repos/add -H 'Content-Type: application/json' \
  -d '{"path": "/path/to/repo", "name": "my-repo"}'
curl -X POST localhost:3000/reindex -H 'Content-Type: application/json' \
  -d '{"repo": "<repo-id>"}'

# Query via service
CANOPY_SERVICE_URL=http://localhost:3000 canopy query --symbol "Config"

Service mode features:

  • Generation tracking for stale-handle safety.
  • Dirty-file local overlay merge for freshness.
  • Handle metadata (source, commit_sha, generation).

Configuration

Create .canopy/config.toml in your repo:

[core]
default_result_limit = 20

[indexing]
default_glob = "**/*.{ts,tsx,js,jsx,py,rs,go}"
preview_bytes = 100
ttl = "24h"

[ignore]
patterns = ["node_modules", ".git", "dist", "build", "__pycache__"]

Architecture

+-----------------+
| canopy-service  |  HTTP service for multi-repo indexing
+-----------------+
| canopy-mcp      |  MCP server for Claude Code
| canopy-cli      |  Command-line interface
+-----------------+
| canopy-client   |  Shared runtime (service client, dirty overlay, merge, predict)
+-----------------+
| canopy-core     |  Indexing and query engine
|  - index/       |  SQLite FTS5 + symbol cache + pipeline + file discovery
|  - parse.rs     |  Tree-sitter parsing
|  - query.rs     |  Query DSL, execution, evidence packs
|  - handle.rs    |  HandleSource + generation metadata
|  - error.rs     |  CanopyError + ErrorEnvelope (shared boundary contract)
+-----------------+

Benchmarking

No canonical benchmark claims are published yet.

For local evaluation:

# Multi-agent comparison
MODE=compare COMPARE_MODES="baseline canopy canopy-service" \
AGENTS=4 MAX_TURNS=5 INDEX_TIMEOUT=1200 \
./benchmark/run-swarm-test.sh /path/to/repo

# Single-agent A/B
./benchmark/run-ab-test.sh /path/to/repo

Detailed protocol, metric definitions, and troubleshooting live in docs/benchmarking.md.

Token metric interpretation:

  • Reported tokens: tokens billed in the run summary.
  • Effective tokens: reported tokens plus cache-read tokens (proxy for total context consumed by the agent loop).
  • Cache-read tokens can dominate long loops, so reported-only views may hide retrieval churn.

Design principles, anti-drift guardrails, and divergence logging live in docs/design-anchors.md.

Compatibility

All crates in this workspace are versioned together and released as a unit. Cross-crate contracts (e.g., ErrorEnvelope, QueryParams, Handle) live in canopy-core and are re-exported by downstream crates.

Component pair Contract Breakage signal
canopy-service ↔ canopy-client JSON over HTTP; ErrorEnvelope, QueryResult, EvidencePack schemas Service integration tests (canopy-client/tests/service_integration.rs)
canopy-mcp ↔ canopy-client ClientRuntime public API MCP build + unit tests
canopy-cli ↔ canopy-client ClientRuntime public API CLI build + unit tests
canopy-client ↔ canopy-core RepoIndex, QueryParams, Handle, EvidencePack Client unit tests

Service mode requires matching canopy-service and canopy-client versions. Running mismatched versions may produce stale_generation or schema errors.

Quality Gates

Behavior Gate Location
Core indexing + query 59 unit tests canopy-core
Client runtime, dirty overlay, merge 32 unit tests canopy-client
Service routes, evidence, state 20 unit tests canopy-service
End-to-end service lifecycle 5 integration tests canopy-client/tests/, canopy-service/tests/
Symbol cache consistency test_symbol_cache_by_file_consistency canopy-core
Pipeline vs sequential indexing test_pipeline_path_indexes_large_batch, test_sequential_path_indexes_small_batch canopy-core
Dirty-file merge correctness test_dirty_merge_* canopy-client

Run cargo test to execute all gates. No benchmark quality gates are published yet; see docs/benchmarking.md for evaluation methodology.

License

MIT

About

Token-efficient codebase intelligence for Claude Code

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 1

v0.3.0 Latest
Feb 14, 2026

Packages

 
 
 

Contributors

Languages