feat(core): improve real-repo analysis resilience

Merge validated real-repo analysis resilience improvements from PR #50.

Author: bntvllnt

README.md

@@ -26,6 +26,7 @@ Common workflows:
 
 ```bash
 npx codebase-intelligence hotspots ./src --metric complexity --limit 10
+npx codebase-intelligence opportunities ./src --limit 10
 npx codebase-intelligence impact ./src parseCodebase
 npx codebase-intelligence dead-exports ./src --limit 20
 npx codebase-intelligence changes ./src --json
@@ -55,7 +56,7 @@ claude mcp add -s user -t stdio codebase-intelligence -- npx -y codebase-intelli
 
 ## Features
 
-- **17 CLI commands** for architecture analysis, dependency impact, dead code detection, search, CI rules, and agent setup
+- **18 CLI commands** for architecture analysis, dependency impact, improvement opportunities, dead code detection, search, CI rules, and agent setup
 - **Machine-readable JSON output** (`--json`) for automation and CI pipelines
 - **Auto-cached index** in `.code-visualizer/` for fast repeat queries
 - **11 architectural metrics** — PageRank, betweenness, coupling, cohesion, tension, churn, complexity, blast radius, dead exports, test coverage, escape velocity
@@ -64,7 +65,7 @@ claude mcp add -s user -t stdio codebase-intelligence -- npx -y codebase-intelli
 - **Process tracing** — detect entry points and execution flows through the call graph
 - **Community detection** — Louvain clustering for natural file groupings
 - **Agent adoption** — `init` writes per-agent instruction files + installs a skill so AI agents query CI before grep/read
-- **MCP parity (secondary)** — same analysis and rules gate available as 16 MCP tools, 2 prompts, and 3 resources
+- **MCP parity (secondary)** — same analysis and rules gate available as 17 MCP tools, 2 prompts, and 3 resources
 
 ## Installation
 
@@ -100,6 +101,7 @@ codebase-intelligence <command> <path> [options]
 | `modules` | Module architecture + cross-dependencies |
 | `forces` | Cohesion/tension/escape-velocity analysis |
 | `dead-exports` | Unused export detection |
+| `opportunities` | Ranked code-quality and refactoring opportunities |
 | `groups` | Top-level directory groups + aggregate metrics |
 | `symbol` | Callers/callees and symbol metrics |
 | `impact` | Symbol-level blast radius |
@@ -119,6 +121,8 @@ codebase-intelligence <command> <path> [options]
 | `--metric <m>` | Select ranking metric for `hotspots` |
 | `--scope <s>` | Select git diff scope for `changes`: `staged`, `unstaged`, `all` |
 
+The scanner always excludes common generated and agent-workspace directories such as `.code-visualizer/`, `.next/`, `dist/`, `coverage/`, `.worktrees/`, and `.claude/worktrees/`.
+
 For full command details, see [docs/cli-reference.md](docs/cli-reference.md).
 
 ## Agent Adoption
@@ -304,6 +308,8 @@ pnpm test         # vitest
 pnpm lint         # eslint
 pnpm typecheck    # tsc --noEmit
 pnpm build        # production build
+pnpm verify:cli-real        # default real-repo CLI matrix
+pnpm verify:cli-real:heavy  # large /home/ubuntu repo matrix
 ```
 
 ## License

docs/architecture.md

@@ -22,8 +22,8 @@ Core (shared computation)
   | result builders used by both MCP and CLI
   v
 MCP (stdio)                    CLI (terminal/CI)
-  | 16 tools, 2 prompts,        | 5 commands: overview, hotspots,
-  | 3 resources for LLMs        | file, search, changes + --json
+  | 17 tools, 2 prompts,        | 18 commands with text + JSON
+  | 3 resources for LLMs        | output for humans and CI
 ```
 
 ## Module Map
@@ -37,13 +37,14 @@ src/
   core/index.ts        <- Shared result computation (MCP + CLI)
   config/index.ts      <- Config discovery + zod validation
   rules/index.ts       <- Rules engine + registry (check command + MCP check tool)
-  mcp/index.ts         <- 16 MCP tools for LLM integration
+  mcp/index.ts         <- 17 MCP tools for LLM integration
   mcp/hints.ts         <- Next-step hints for MCP tool responses
   impact/index.ts      <- Symbol-level impact analysis + rename planning
   search/index.ts      <- BM25 search engine
   process/index.ts     <- Entry point detection + call chain tracing
   community/index.ts   <- Louvain clustering
   persistence/index.ts <- Graph export/import to .code-visualizer/
+  persistence/cache-key.ts <- Cache signature from HEAD, worktree content, CLI version, parser settings
   install/index.ts     <- Agent adoption: managed-block engine + per-agent file targets + skill
   server/graph-store.ts <- Global graph state (shared by CLI + MCP)
   cli.ts               <- Entry point, CLI commands + MCP fallback
@@ -66,17 +67,20 @@ analyzeGraph(builtGraph, parsedFiles)
      }
 
 startMcpServer(codebaseGraph)
-  -> stdio MCP server with 16 tools, 2 prompts, 3 resources
+  -> stdio MCP server with 17 tools, 2 prompts, 3 resources
 ```
 
 ## Key Design Decisions
 
 - **Dual interface**: MCP stdio for LLM agents, CLI subcommands for humans/CI. Both consume `src/core/`.
 - **graphology**: In-memory graph with O(1) neighbor lookup. PageRank and betweenness computed via graphology-metrics.
 - **Batch git churn**: Single `git log --all --name-only` call, parsed for all files. Avoids O(n) subprocess spawning.
+- **Monorepo import resolution**: Root `tsconfig.json` path aliases and local `package.json` package names resolve to source files before graph construction.
+- **Large repo fallback**: Above 1500 TypeScript files, the parser uses AST-only extraction to keep file/import/export analysis available without TypeScript program OOM.
 - **Dead export detection**: Cross-references parsed exports against edge symbol lists. May miss `import *` or re-exports (known limitation).
 - **Graceful degradation**: Non-git dirs get churn=0, no-test codebases get coverage=false. Never crashes.
-- **Graph persistence**: CLI commands always cache the graph index to `.code-visualizer/`. MCP mode (`codebase-intelligence <path>`) requires `--index` to persist the cache.
+- **Built-in scanner excludes**: Generated indexes, build outputs, coverage, framework caches, and agent worktrees are skipped before TypeScript program creation.
+- **Graph persistence**: CLI commands always cache the graph index to `.code-visualizer/`. Cache reuse requires matching HEAD, dirty/untracked file-content fingerprint, CLI version, and parser cache settings. MCP mode (`codebase-intelligence <path>`) requires `--index` to persist the cache.
 
 ## Adding a New Metric
 

docs/cli-reference.md

@@ -1,6 +1,6 @@
 # CLI Reference
 
-17 commands for terminal and CI use. The 15 analysis commands have full parity with MCP tools and auto-cache the index to `.code-visualizer/`; `check` runs the rules gate; `init` sets up agent adoption.
+18 commands for terminal and CI use. The 16 analysis commands have full parity with MCP tools and auto-cache the index to `.code-visualizer/`; `check` runs the rules gate; `init` sets up agent adoption.
 
 ## Commands
 
@@ -12,7 +12,7 @@ High-level codebase snapshot.
 codebase-intelligence overview <path> [--json] [--force]
 ```
 
-**Output:** file count, function count, dependency count, modules (path, files, LOC, coupling, cohesion), top 5 depended files, avg LOC, max depth, circular dep count.
+**Output:** file count, function count, dependency count, analysis mode/call graph precision, modules (path, files, LOC, coupling, cohesion), top 5 depended files, avg LOC, max depth, circular dep count.
 
 ### hotspots
 
@@ -94,7 +94,17 @@ Find unused exports across the codebase.
 codebase-intelligence dead-exports <path> [--module <module>] [--limit <n>] [--json] [--force]
 ```
 
-**Output:** dead export count, files with unused exports, summary.
+**Output:** dead export count, files with unused exports, confidence, package-entrypoint evidence, summary.
+
+### opportunities
+
+Rank code quality and refactoring opportunities for AI agents.
+
+```bash
+codebase-intelligence opportunities <path> [--limit <n>] [--json] [--force]
+```
+
+**Output:** ranked opportunities with kind, priority, confidence, score, target, evidence, and suggested follow-up commands.
 
 ### groups
 
@@ -198,7 +208,7 @@ codebase-intelligence init [path] [--agents <list>] [--all] [--skill] [--yes] [-
 | `--json` | All commands | Output stable JSON to stdout |
 | `--force` | All commands | Re-parse even if cached index matches HEAD |
 | `--metric <m>` | hotspots | Metric to rank by (default: coupling) |
-| `--limit <n>` | hotspots, search, dead-exports, processes | Max results |
+| `--limit <n>` | hotspots, search, dead-exports, opportunities, processes | Max results |
 | `--scope <s>` | changes | Git diff scope: staged, unstaged, all |
 | `--depth <n>` | dependents | Max traversal depth (default: 2) |
 | `--cohesion <n>` | forces | Min cohesion threshold (default: 0.6) |
@@ -221,7 +231,11 @@ codebase-intelligence init [path] [--agents <list>] [--all] [--skill] [--yes] [-
 
 ## Behavior
 
-**Auto-caching:** First CLI invocation parses the codebase and saves the index to `.code-visualizer/`. Subsequent commands use the cache if `git HEAD` hasn't changed. Add `.code-visualizer/` to `.gitignore`.
+**Auto-caching:** First CLI invocation parses the codebase and saves the index to `.code-visualizer/`. Subsequent commands use the cache only when `git HEAD`, dirty/untracked file contents under the analyzed path, the CLI version, and parser cache settings match. Add `.code-visualizer/` to `.gitignore`.
+
+**Default scanner excludes:** The parser always skips `.git`, `node_modules`, `.code-visualizer`, `.next`, `dist`, `coverage`, `.turbo`, `.cache`, `.worktrees`, and `.claude/worktrees`, even if the target repo has no matching `.gitignore` entry.
+
+**Large repo mode:** Repos above 1500 TypeScript files use a lightweight AST parser by default to avoid TypeScript program OOM. File/import/export/dependency metrics remain available; type-resolved call graph details are reduced. Set `CBI_FULL_PROGRAM_FILE_LIMIT=<n>` to tune the cutoff.
 
 **stdout/stderr:** Results go to stdout. Progress messages go to stderr. Safe for piping (`| jq`, `> file.json`).
 
@@ -234,4 +248,4 @@ codebase-intelligence init [path] [--agents <list>] [--all] [--skill] [--yes] [-
 - `--index` — persist graph index to `.code-visualizer/` (CLI auto-caches, MCP requires this flag)
 - `--status` — print index status and exit
 - `--clean` — remove `.code-visualizer/` index and exit
-- `--force` — re-index even if HEAD unchanged
+- `--force` — re-index even if the cache signature matches

docs/data-model.md

@@ -76,6 +76,9 @@ FileMetrics {
   cyclomaticComplexity: number  // Avg complexity of exports
   blastRadius: number           // Transitive dependent count
   deadExports: string[]         // Unused export names
+  totalExports: number          // Named export count used as dead-export denominator
+  isPackageEntrypoint: boolean  // package.json exports/main/types/bin point here
+  packageEntrypointReason: string // package.json field/path evidence
   isTestFile: boolean           // Whether this file is a test file
 }
 
@@ -107,6 +110,9 @@ CodebaseGraph {
     totalFunctions: number
     totalDependencies: number
     circularDeps: string[][]  // Each cycle = array of file paths
+    analysisMode: "full-program" | "ast-only"
+    callGraphPrecision: "type-resolved" | "syntax-only"
+    fullProgramFileLimit: number
   }
 }
 ```

docs/mcp-tools.md

@@ -1,13 +1,13 @@
 # MCP Tools Reference
 
-16 tools available via MCP stdio.
+17 tools available via MCP stdio.
 
 ## 1. codebase_overview
 
 High-level summary of the entire codebase.
 
 **Input:** `{ depth?: number }`
-**Returns:** totalFiles, totalFunctions, totalDependencies, modules (sorted by size), topDependedFiles (top 5 by fanIn), globalMetrics (avgLOC, maxDepth, circularDepCount)
+**Returns:** totalFiles, totalFunctions, totalDependencies, modules (sorted by size), topDependedFiles (top 5 by fanIn), metrics (avgLOC, maxDepth, circularDeps), analysis (mode, callGraphPrecision, fullProgramFileLimit)
 
 **Use when:** First exploring a codebase. "What does this project look like?"
 **Not for:** Module details (use get_module_structure) or data flow (use analyze_forces).
@@ -17,7 +17,7 @@ High-level summary of the entire codebase.
 Detailed context for a single file.
 
 **Input:** `{ filePath: string }` (relative path)
-**Returns:** path, loc, exports, imports (with symbols, isTypeOnly, weight), dependents (with symbols, isTypeOnly, weight), metrics (all FileMetrics including churn, complexity, blastRadius, deadExports, hasTests, testFile)
+**Returns:** path, loc, exports, imports (with symbols, isTypeOnly, weight), dependents (with symbols, isTypeOnly, weight), metrics (all FileMetrics including churn, complexity, blastRadius, deadExports, totalExports, package-entrypoint metadata, hasTests, testFile)
 
 **Path normalization:** Backslashes are normalized to forward slashes. The exact graph path is tried first; if not found, common prefixes (`src/`, `lib/`, `app/`) are stripped once from the leading position and retried. If the file is not found, the error includes up to 3 suggested similar paths.
 
@@ -70,12 +70,24 @@ Architectural force analysis — module health, misplaced files, bridge files.
 Find unused exports across the codebase.
 
 **Input:** `{ module?: string, limit?: number }` (default limit: 20)
-**Returns:** totalDeadExports, files (with path, module, deadExports[], totalExports), summary
+**Returns:** totalDeadExports, files (with path, module, deadExports[], totalExports, confidence, package-entrypoint metadata), summary
+
+Package public entrypoints from `exports`, `main`, `types`, and `bin` are reported as low confidence because external consumers may use them.
 
 **Use when:** Cleaning up dead code, reducing API surface.
 **Not for:** Finding used exports (use file_context).
 
-## 8. get_groups
+## 8. find_opportunities
+
+Rank code quality and refactoring opportunities for AI agents.
+
+**Input:** `{ limit?: number }` (default limit: 20)
+**Returns:** totalOpportunities, opportunities[] (rank, kind, target, title, priority, score, confidence, why, evidence[], suggestedCommands[]), summary
+
+**Use when:** "What should I improve?" "Find refactoring opportunities." "Which files need tests?"
+**Not for:** Raw metric lists only (use find_hotspots or analyze_forces).
+
+## 9. get_groups
 
 Top-level directory groups with aggregate metrics.
 
@@ -85,7 +97,7 @@ Top-level directory groups with aggregate metrics.
 **Use when:** "What are the main areas of this codebase?" High-level grouping overview.
 **Not for:** Detailed module metrics (use get_module_structure).
 
-## 9. symbol_context
+## 10. symbol_context
 
 Callers, callees, and importance metrics for a function, class, or method.
 
@@ -95,7 +107,7 @@ Callers, callees, and importance metrics for a function, class, or method.
 **Use when:** "Who calls X?" "Trace this function." "What depends on this symbol?"
 **Not for:** Text search (use search) or file-level dependencies (use get_dependents).
 
-## 10. search
+## 11. search
 
 Search files and symbols by keyword.
 
@@ -105,7 +117,7 @@ Search files and symbols by keyword.
 **Use when:** "Find files related to auth." "Where is getUserById defined?"
 **Not for:** Structured call graph queries (use symbol_context).
 
-## 11. detect_changes
+## 12. detect_changes
 
 Detect changed files from git diff with risk metrics.
 
@@ -115,7 +127,7 @@ Detect changed files from git diff with risk metrics.
 **Use when:** Starting a review, triaging changes, "what changed?"
 **Not for:** Symbol-level impact (use impact_analysis).
 
-## 12. impact_analysis
+## 13. impact_analysis
 
 Symbol-level blast radius with depth-grouped risk labels.
 
@@ -125,7 +137,7 @@ Symbol-level blast radius with depth-grouped risk labels.
 **Use when:** "What breaks if I change getUserById?" Symbol-level impact assessment.
 **Not for:** File-level dependencies (use get_dependents).
 
-## 13. rename_symbol
+## 14. rename_symbol
 
 Read-only reference finder for rename planning.
 
@@ -135,7 +147,7 @@ Read-only reference finder for rename planning.
 **Use when:** Planning a rename, finding all usages of a symbol.
 **Not for:** Call graph analysis (use symbol_context).
 
-## 14. get_processes
+## 15. get_processes
 
 Trace execution flows from entry points through the call graph.
 
@@ -145,7 +157,7 @@ Trace execution flows from entry points through the call graph.
 **Use when:** "How does this app start?" "Trace request flow." "What are the entry points?"
 **Not for:** Static file dependencies (use get_dependents).
 
-## 15. get_clusters
+## 16. get_clusters
 
 Community-detected clusters of related files.
 
@@ -155,7 +167,7 @@ Community-detected clusters of related files.
 **Use when:** "What files are related?" "Find natural groupings." Discovering emergent groupings that differ from directory structure.
 **Not for:** Directory-based modules (use get_module_structure).
 
-## 16. check
+## 17. check
 
 Run the configurable rules engine and gate on findings.
 
@@ -192,6 +204,8 @@ Rules: `no-comments` (off by default), `no-circular-deps` (error), `no-dead-expo
 | "What breaks if I change function X?" | `impact_analysis` |
 | "What are the riskiest files?" | `find_hotspots` (coupling, churn, or blast_radius) |
 | "Which files need tests?" | `find_hotspots` (coverage) |
+| "What should I improve first?" | `find_opportunities` |
+| "Find refactoring opportunities." | `find_opportunities` |
 | "What can I safely delete?" | `find_dead_exports` |
 | "How are modules organized?" | `get_module_structure` |
 | "What's architecturally wrong?" | `analyze_forces` |