architecture.md

How CALM works

This is the technical deep-dive the main README only summarizes — multi-tier indexing, the SCIP/LSP overlay system, search, the edit safety net, concurrency, self-grading, memory, and the sanitization layer. If you just want to know what CALM does for you and how to install it, the README is a shorter, better place to start; come back here once you're deciding whether to depend on it, or you want to know exactly how a specific guarantee is implemented.

Design philosophy

CALM isn't a pile of MCP tools bolted together — it's designed as a map and an active co-pilot for the agent actually holding the wheel, not a dashboard for a human watching from the sidelines.

• Every response carries suggested_next. The agent is rarely left guessing what step comes next — the tool that just ran tells it.
• The genuinely risky steps are hard-gated, not just recommended. edit_context before any edit, diff_impact before any commit — these are enforced, not suggested. Everything lower-stakes just nudges; the agent keeps its own judgment where the cost of being wrong is low.
• The signals are proactive, not something the agent has to ask for. fitness_report, session_context's pending_diff_impact / possibly_stuck, repo_overview's memory_notes_count — the agent never has to remember "did I already check impact?" or notice on its own "am I going in circles?". CALM answers before it's asked.

The end goal is reduced cognitive load: the agent spends its budget on the work that actually creates value, not on managing its own navigational bookkeeping.

Multi-tier indexing

• 6 Tier-0 languages — Python, TypeScript, JavaScript, Java, Rust, Go — get full tree-sitter AST parsing, a real call graph, an import graph, and multi-tier resolution, always on, no feature flag required.
• 18 Tier-0.5 languages — full tree-sitter AST parsing with call-graph and import resolution, gated behind Cargo features:
  • On by default (tier0-5 feature bundle): C, C++, C#, Ruby, PHP, Shell, R.
  • Opt-in, one feature flag each: Kotlin, Swift (lang-kotlin/lang-swift), plus Scala, Dart, Lua, Elixir, Haskell, OCaml, Zig, PowerShell, Groovy — the 9 languages added in the 25-language expansion's Phase C batch. Falls back to regex/line-scan symbol extraction (no call graph) only when the matching grammar feature isn't compiled in.
  • Dart is the one exception worth calling out by name: it parses symbols cleanly but produces zero call edges, because its tree-sitter grammar has no call-expression node — a documented limitation, not a bug.
• SQL gets its own standalone indexer (sqlparser, real grammar parsing, not regex) — extracts tables/views/procedures accurately across Postgres/MySQL/SQL Server dialects, but stops short of a call graph, since "calls" isn't a coherent concept across SQL dialects the way it is for the languages above.
• Incremental watcher — only changed files get re-parsed (FNV-1a content hash diff); the call graph rebuilds incrementally, parallelized with rayon. calm serve picks incremental reindex automatically whenever an index already exists.

A call graph you can actually trust

• Every edge carries a confidence label — resolved / inferred / formal / textual (plus ambiguous/unresolved fallback tiers when a call site's target genuinely can't be pinned down) — so an agent knows when it's looking at a sure thing versus a best guess.
• SCIP overlay — formal, compiler-grade ground truth, 9 providers spanning 12 languages: Rust (rust-analyzer), Go (scip-go, including multi-module go.work workspaces — indexed once per member module, then rebased into one graph, since scip-go itself has no native workspace flag), Python (scip-python), JavaScript + TypeScript (scip-typescript), Java (scip-java — also indexes Kotlin in the same pass, since scip-java bundles a kotlinc plugin for mixed Gradle/Maven modules), C# (scip-dotnet), PHP (scip-php), Ruby (scip-ruby/Sorbet), and C/C++ (scip-clang, live-verified 2026-07-15). Each language is a data-driven ScipProvider entry (crates/calm-core/src/scip/provider.rs), not a copy-pasted module — adding another language is one table row. Every provider auto-detects its own binary and silently sits out if it isn't there (zero behavior change on a machine missing that toolchain), runs under a hard timeout, and caches against a per-language fingerprint (lockfile/build-file hash + toolchain + dirty source keys) so an unchanged project never re-pays the cost. Verification maturity varies honestly by language: .github/workflows/scip-nightly.yml runs one independent job per provider (restructured 2026-07-15 from a single monolithic job, so one provider's install failure can no longer suppress test evidence for the others) — all 9 providers (Rust/Go/Python/JavaScript/Java/C#/Ruby/C++/PHP) were live-verified against a real installed binary during the 2026-07-15 restructure, but as of that same day the automated nightly cron had not yet completed a post-restructure run (check gh run list --workflow=scip-nightly.yml for current status) — "works against a real binary" and "nightly CI is green" are two separate claims, don't conflate them until a run confirms the latter. PHP's own path was the last one fixed, and not with a security-posture workaround: Packagist's only scip-php release (v0.0.2, 2023) has two independent problems — a Composer advisory block on its google/protobuf range, and a separate RuntimeException: Invalid scip-php vendor directory crash (github.com/davidrjenni/scip-php#235) that's already fixed upstream (PR #797, merged 2026-03-28) but never cut into a release or rebuilt into the davidrjenni/scip-php Docker image (reported: github.com/davidrjenni/scip-php/issues/862). Both disappear installing from that post-fix commit directly instead of from Packagist — main's own composer.json already requires a patched google/protobuf range, so the advisory question doesn't even arise. Pinned to an exact commit (not floating main) for reproducibility; bump it once upstream ships a real release.
• LSP overlay — a second, complementary path to formal edges: for Go (gopls) and C/C++ (clangd) — plus Rust's rust-analyzer on a live-session path, distinct from its batch SCIP export — a real language server resolves ambiguous/textual call sites interactively, on demand. This doesn't run automatically (policy defaults to on_demand, and always will unless per-save latency is measured safe — see ADR-0004); trigger it explicitly with the lsp_refresh tool or calm scip-run/equivalent CLI.
• Trigger either overlay on demand with calm scip-run --lang <rust|go|python|javascript|java|csharp|php|ruby|c|all> or the scip_refresh/lsp_refresh MCP tools; calm index --scip-file <path.scip> --sub-root <dir> ingests a pre-built SCIP index instead, for CI/sandboxed runs with no network access to install an indexer.
• Trace output stays readable at scale. A real hub symbol can have dozens of callers — callers/callees/edit_context put non-test callers first and cap the visible list at a configurable size (the true total is still reported alongside it), so the one production call site isn't buried behind sixty near-identical test fixtures. A repeat call on an unchanged symbol can skip resending the list entirely via if_none_match/etag conditional-fetch, the same pattern source() already used — measured up to a ~60% token reduction on a real hub symbol in this repo's own graph, with the pre-edit risk assessment always computed from the full list before it's ever truncated.
• Graph metrics — coreness (k-core) and is_hub — flag the symbols central enough that touching them is inherently higher-risk. repo_overview.core_symbols reuses the same metric to sketch the architecture's "skeleton" on the very first call (inspired by Aider's PageRank repo-map, but built on a metric CALM already computes rather than a separate pass).

Search that actually finds things

• Full-text + semantic search, fused — FTS5 (BM25) combined with semantic embeddings (model2vec-rs, pure Rust, no ONNX) via a 3-way Reciprocal Rank Fusion (text + symbol-identity vector + code-body-chunk vector) — finds relevant code even when the query doesn't share a token with the symbol name. KNN is a brute-force cosine scan in pure Rust with an in-RAM cache — no C vector-search extension, so it behaves identically on every release platform (the previous sqlite-vec dependency didn't compile on musl libc, which silently killed semantic search on Linux/Docker builds). The default model (minishlab/potion-code-16M, MIT-licensed — from MinishLab, the same lab behind Semble, one of the tools benchmarked in the README's Measured against the tools that came before it) is vendored straight into the binary: build.rs fetches the weights from Hugging Face and SHA256-verifies them at compile time (cached in the working tree afterward — not Git LFS, that mechanism was removed 2026-07-12), then embeds them via include_bytes! — no network needed at runtime for the default case; a failed compile-time fetch falls back to a placeholder plus a one-time Hugging Face download at runtime instead, unless you explicitly opt out (allow_network_fallback) to keep a strict zero-network guarantee.
• Real grep/glob, straight off disk — search(kind="grep") uses actual regex + glob filtering through a .gitignore-respecting walker, bypassing the index entirely — so it reaches files the indexer never parses (Cargo.toml, docs/*.md) too, each match enriched with its surrounding symbol when one exists.
• Scores you can actually read — search/locate round RRF/similarity scores to 4 decimal places before returning them (0.01639344262295082 → 0.0164) — purely representational, doesn't change ranking.
• Noise-penalty ranking — results living in test/generated/example files are scored down when an equivalent real-implementation result exists, so the actual code surfaces first instead of getting buried under a same-named test fixture.

Editing with an actual safety net

• edit_lines/edit_symbol — the one write path, working on any tracked file (not just parsed symbols). A content-hash conflict guard (FNV-1a) on the exact line range rejects stale writes and hands back the current hash/content to re-read; multiple hunks in one call apply bottom-up so offsets never drift between them.
• Syntax-validated before it ever touches disk — tree-sitter checks the result parses cleanly; a write that would introduce a syntax error is refused outright, nothing gets written.
• Hub and high-fan-in symbols need three things, not just one — edit_context must have actually been called for that exact symbol this session (not a prior session, not a stale review), confirm:true, and a reason string that cites a real caller name edit_context itself returned, not a generic phrase like "this looks safe". A policy only a tool with a real call graph — and a memory of what it just showed you — can enforce.
• Atomic writes, immediate reindex — temp file + fsync + rename, then reindexed synchronously (not waiting on the file watcher); the response comes back with post-edit risk/callers, like a miniature diff_impact.
• Hook-enforced, not just documented — under Claude Code, .claude/hooks/calm-nudge.sh actually blocks the first Edit of a session until edit_context has been called, and blocks git commit/git push if files changed since the last diff_impact. session_context's pending_diff_impact gives the same signal on any other MCP client.

Concurrency & reliability

Running CALM from more than one editor session on the same repo used to mean N independent indexers, N SQLite connections, and N copies of the embedding model in RAM — real pain found by pointing CALM's own instrumentation at this repo mid-session. That's been closed from multiple directions:

• Cross-process edit lock — an OS flock on .calm/edit.lock, layered underneath the in-process hash check, closes a narrow TOCTOU window where two separate processes could both pass a stale-hash check and the second write would silently discard the first.
• Single-instance indexing lock, with promotion — only one calm serve process per project root ever runs the background indexer/watcher; a losing process auto-promotes to owner if the current owner exits mid-session, instead of a second editor session being stuck read-only forever.
• A real SIGTERM watchdog — a raw kernel alarm() guarantees the process exits even when the MCP transport's stdio-read thread is blocked in an uncancellable OS read, a bug that a purely async watchdog silently never caught.
• A shared-daemon model, on by default — calm serve --listen unix:PATH runs one owning daemon; calm connect gives every other session a lightweight forwarder over the same socket instead of its own full process, with automatic stale-build detection (daemon.meta) that respawns the daemon after a rebuild instead of silently serving an old binary. scripts/mcp-launcher.sh (and therefore the npm/plugin distribution) defaults to this on Unix whenever no extra launcher args are in play — falls back to the original one-process-per-session calm serve for any custom invocation, or set CI_MCP_LAUNCHER_NO_DAEMON=1 to opt out entirely.
• Concurrent-agent awareness — under the shared daemon, session_context.other_active_sessions reports every other connection sharing that socket right now (the file it last touched, when, how many tool calls), so an agent can notice "someone else is already working in this file" before stepping on the same area — not full multi-agent coordination, just making concurrent sessions visible instead of invisible.

The codebase grading itself

• calm fitness-check / fitness_report — 11 metrics (hub_count, hub_pct, avg_coreness, dead code, hotspot risk, edge coverage, cyclomatic complexity, average distance, architecture-boundary violations, ambiguous boundaries, and config drift) checked against thresholds in thresholds.toml, queryable mid-session or as a CI gate.
• Coverage-aware dead-code detection — auto-detects lcov / .coverage / Go coverage.out / Cobertura XML at startup and folds real runtime coverage into dead_code_confidence, so code a test actually exercises at runtime doesn't get flagged just because the static call graph missed the call site. scripts/gen-coverage.sh generates one on demand for this repo itself.
• Architecture boundaries — [[boundaries]] — declare "module A must not import module B" directly in thresholds.toml, matched by path prefix against the real import graph; every violation is reported with the actual offending file pair, not just a count.
• Doc-drift detection — [config_drift] — flags file-path references inside declared docs that no longer point at anything real, so a design doc doesn't quietly keep describing a file that was deleted three refactors ago.

An agent that remembers, and knows when it's stuck

• remember/recall — durable, interpretive notes (an architecture decision, a gotcha) keyed by topic, surviving restarts — distinct from session_context, which only tracks in-session navigation and resets on restart.
• Notes surface themselves, without a separate recall call — edit_context/locate automatically attach any rememberd note that references the file in play (related_notes). On a hub file a note only qualifies if its text names the exact symbol (specificity: "symbol"); a plain file-level match is used only on smaller, non-hub files, so one old note doesn't bury every symbol in a large file forever. A note that trips the same prompt-injection heuristic source's content_warning uses is left out of this automatic surface — still fully readable via an explicit recall.
• pattern_debt_register/pattern_debt_status — found the same bug in more than one place? Anchor it by the symbol's qualified_name (survives line-shifting edits elsewhere in the file, unlike a raw path+line) and baseline the duplicate count with search(kind="similar"); re-check later and get back open, resolved, or anchor_lost (the anchor symbol itself was renamed/removed — reported honestly, never silently counted as fixed).
• Git co-change mining — edit_context mines git log for files that historically change alongside the one being edited despite no import/call relationship (a model and its migration, say) — a coupling signal the static graph can't see on its own.
• Session progress signal — session_context.possibly_stuck flags 10+ tool calls with no new file/symbol touched; informational only, the decision to break the loop stays with the host (e.g. Claude Code's /goal).
• MCP Prompts — review_symbol, debug_symbol, onboard_area package a full multi-step workflow into one slash-command-style call.

Honest about its own freshness

• Index state machine surfaced everywhere — scanning → parsing → building_edges → ready, so an agent never mistakes stale data for current.
• Build-freshness check — calm doctor compares the commit the running binary was built from against the repo's current HEAD; scripts/mcp-launcher.sh checks source mtimes before trusting an existing target/{debug,release}/calm, rebuilding rather than silently serving a stale binary.

Safe by default

• Output sanitization — source/understand redact credential-shaped text (PEM keys, GitHub/AWS/Slack tokens, JWTs, password assignments) before it's ever returned, and flag a content_warning when code contains prompt-injection-shaped text — flagged, never silently altered, since a false positive there would corrupt real code. The heuristic (calm_core::sanitize, 19 labeled patterns) covers plain-English phrasing ("ignore previous instructions"), chat-template role-marker spoofing (ChatML <|im_start|>, [INST]/[SYS] brackets, fake system:/Markdown-heading role markers), fake tool/turn-boundary tags (</tool_result>, <system>), jailbreak/persona-override phrasing, exfiltration phrasing (prompt/secret-reveal requests), zero-width Unicode obfuscation, and a first pass at Vietnamese-language equivalents — deliberately excludes anything with real false-positive risk (e.g. generic tag-density scoring, homoglyph detection) rather than guess.
• scan_text — the same detection, on demand, for anything that didn't come through the index. source/understand's content_warning only covers indexed source; every tool's own output is separately scanned (advisory-logged, not surfaced) at one central choke point (timed_tool). Neither covers content an agent fetches itself — a WebFetch/WebSearch result, a subagent's report. scan_text closes that gap: point it at any text and get the same labeled hits back, entirely local and regex-based — no dependency on a hosted LLM safety classifier being available, so it keeps working even when that classifier isn't.
• Local-first — no outbound calls for the code/data path when the valid embedding weights are already embedded. If the build-time fetch failed, the runtime model fallback is enabled by default via semantic_search.allow_network_fallback; set it to false for strict offline operation. The other exception is opt-in OpenTelemetry span export (below), which is off by default.

Observability (optional)

calm can export the spans it already emits (mcp_tool_call, tool-execution timing) to an OpenTelemetry collector, entirely opt-in and off by default — see crates/calm-cli/src/otel.rs.

• Built only with --features otel, and even then the pipeline is never constructed unless OTEL_EXPORTER_OTLP_ENDPOINT is also set at runtime — no exporter, no tracer provider, no background export task, and no network I/O when the env var is absent, verified by crates/calm-cli/tests/otel_gate.rs (otel_layer returns None).
• HTTP transport, not gRPC — pinned to opentelemetry-otlp's http-proto feature (POSTs over reqwest), not grpc-tonic, so no gRPC channel is ever opened at runtime. (tonic/tonic-prost still compile in as transitive build dependencies of opentelemetry-proto regardless of this feature choice — that crate doesn't gate them out of the dependency graph, only out of opentelemetry-otlp's own default feature set — so this narrows runtime behavior, not full compile-time weight.)
• What leaves the machine when enabled: span attributes — file paths, symbol names, tool names, timing — sent to the collector at OTEL_EXPORTER_OTLP_ENDPOINT. Never source code bodies. See the README's egress note for the exact qualification of the "local-only" claim above.
• Manual smoke test (not run in CI — needs a live collector): run a local otel-collector; build with cargo build --features otel; then export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 and run that binary's calm serve for one session, make a few tool calls, and confirm mcp_tool_call spans arrive at the collector.

---

Back to the README for install instructions, the MCP tool reference, and the benchmark summary.