docs: document v0.18.0 analytics across external and internal guides

aorumbayev · claude · aorumbayev · commit e622d8f447e8 · 2026-04-16T21:24:11.000+02:00
Add coverage for multi-dimensional analytics, intelligent backend
selection, and new MCP tools across all user-facing and contributor
doc surfaces:

External docs:
- reference/mcp-tools.md: new Analytics tools section + readonly tier
- reference/configuration.md: use_recommended_backend setting
- guides/web-dashboard.md: Analytics page section with 4-tab overview
- guides/mcp-setup.md: Analytics tools subsection for MCP clients

Internal docs:
- internal/features/server.md: 9 analytics REST routes documented
- internal/features/mcp.md: new Analytics Tools section + renumbered

Cross-links point to the existing analytics.md guide for deep dives.
Style and formatting match each file's existing conventions.

Co-Authored-By: Claude Opus 4.6 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/docs/guides/mcp-setup.md b/docs/guides/mcp-setup.md
@@ -277,6 +277,20 @@ Add `"--role", "WORKER"`, `"--role", "REVIEWER"`, or `"--role", "ORCHESTRATOR"`
 
 ______________________________________________________________________
 
+## Analytics tools
+
+Kagan exposes its analytics data over MCP so external clients (Claude Desktop, Cursor, Claude Code, etc.) can query agent performance and session activity programmatically. These tools are available to every role (`WORKER`, `REVIEWER`, `ORCHESTRATOR`) and operate on the currently active project — open a project in Kagan first, otherwise they return empty results.
+
+| Tool                         | What it returns                                                                  |
+| ---------------------------- | -------------------------------------------------------------------------------- |
+| `analytics_backend_stats`    | Per-backend session count, success rate, average duration, and retry rate.       |
+| `analytics_session_timeline` | Daily session counts by status. Accepts `days` (default `30`).                   |
+| `analytics_export`           | Combined backend stats + session timeline, ready for dashboards. Accepts `days`. |
+
+See the [Analytics guide](analytics.md) for the full list of metrics, UI surfaces, and export workflows.
+
+______________________________________________________________________
+
 ## Multi-repo
 
 **Create:** Kagan → New Project → add repo paths. First repo = active.
diff --git a/docs/guides/web-dashboard.md b/docs/guides/web-dashboard.md
@@ -15,13 +15,14 @@ kagan web --host 0.0.0.0    # LAN access
 
 ## Activity bar
 
-The left-edge activity bar has three icons:
+The left-edge activity bar has four icons:
 
-| Icon      | Route        | Purpose                              |
-| --------- | ------------ | ------------------------------------ |
-| Board     | `/board`     | Kanban board with drag-and-drop      |
-| Workspace | `/workspace` | Orchestrator-first conversation view |
-| Settings  | `/settings`  | Categorized settings with sidebar    |
+| Icon      | Route         | Purpose                                |
+| --------- | ------------- | -------------------------------------- |
+| Board     | `/board`      | Kanban board with drag-and-drop        |
+| Workspace | `/workspace`  | Orchestrator-first conversation view   |
+| Analytics | `/analytics`  | Agent performance and session metrics  |
+| Settings  | `/settings`   | Categorized settings with sidebar      |
 
 Toggle between Board and Workspace with `Cmd/Ctrl+Shift+W`.
 
@@ -46,6 +47,25 @@ A conversation-first companion to the board, modeled after ChatGPT / Codex Deskt
 - On first visit, if no conversations exist, Kagan creates a blank orchestrator session automatically.
 - The global AI rail does not open on `/workspace`; this route already is the AI surface.
 
+## Analytics
+
+Added in v0.18.0. The `/analytics` page surfaces multi-dimensional metrics across your agent runs, with four tabs:
+
+| Tab              | Content                                                                                                            |
+| ---------------- | ------------------------------------------------------------------------------------------------------------------ |
+| **Backend**      | KPI cards (Total Sessions, Success Rate, Avg Duration, Retry Rate), backend performance table, duration chart, session timeline |
+| **By Role**      | Per-role success rates (Worker / Orchestrator / Reviewer) and a role comparison chart                              |
+| **By Task Type** | Per-task-type success rates and a Backend x Task Type matrix                                                       |
+| **Combined**     | 3D table of Backend x Role x Task Type combinations                                                                |
+
+Top-of-page controls:
+
+- **Time range dropdown**: 7 / 14 / 30 days.
+- **Export**: downloads the current dataset as JSON.
+- **Glossary**: help icon that explains each metric.
+
+See [Analytics & Metrics](./analytics.md) for the full metric definitions, task-type classifier, and access across other surfaces (TUI, CLI, VS Code, MCP).
+
 ## Settings
 
 The settings page uses a left sidebar for category navigation. See [Configuration reference -- Web dashboard settings](../reference/configuration.md#web-dashboard-settings) for the full category list.
diff --git a/docs/internal/features/mcp.md b/docs/internal/features/mcp.md
@@ -111,7 +111,19 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
-## 12. Resources
+## 12. Analytics Tools
+
+Read-only wrappers around `Analytics` in `kagan.core._analytics`, scoped to the active project. All three tools are registered in the WORKER tier so any agent role can query them; they return empty payloads when no project is active.
+
+- `analytics_backend_stats` — per-backend aggregates (count, success rate, avg duration, retry rate) for choosing an agent backend
+- `analytics_session_timeline(days=30)` — daily session counts bucketed by status (completed/failed/cancelled/running/pending) for trend charts
+- `analytics_export(days=30)` — combined `{exported_at, period_days, backend_stats, session_timeline}` snapshot for archival or offline analysis
+
+The richer multi-dimensional analytics (by agent role, by task type, per-task backend recommendation) live only on the REST surface under `/api/analytics/*` (see `src/kagan/server/_analytics_routes.py`). They are intentionally not exposed as MCP tools yet — add them to `toolsets/analytics.py` and register in `_policy.py` if a use case emerges.
+
+______________________________________________________________________
+
+## 13. Resources
 
 - Read-only data endpoints, always available regardless of access mode
 - `kagan://ping` — health check
@@ -122,7 +134,7 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
-## 13. Access Control
+## 14. Access Control
 
 - Three roles: WORKER (board awareness + own-task annotation), REVIEWER (+ verdicts), ORCHESTRATOR (full control)
 - `--role` flag sets the agent role; defaults to ORCHESTRATOR
diff --git a/docs/internal/features/server.md b/docs/internal/features/server.md
@@ -200,6 +200,26 @@ curl -X POST http://localhost:8765/api/plugins/{name}/import
 
 ______________________________________________________________________
 
+## Analytics Routes
+
+Registered in `src/kagan/server/_analytics_routes.py`. All endpoints require an active project context — if none is set, they return an empty list or object rather than erroring. Used by the web dashboard's analytics views and by agents deciding which backend to invoke.
+
+| Endpoint                                                                 | Purpose                                                                                             |
+| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- |
+| `GET /api/analytics/backend-stats`                                       | Per-backend aggregates (sessions, success rate, duration) for the active project.                   |
+| `GET /api/analytics/session-timeline?days=N`                             | Daily session counts over the last `N` days (default 30).                                           |
+| `GET /api/analytics/timeline-summary?days=N`                             | Rolled-up timeline aggregates over `N` days (default 30).                                           |
+| `GET /api/analytics/recommended-backend`                                 | Simple recommendation: the backend with the highest success rate for this project.                  |
+| `GET /api/analytics/export?days=N`                                       | Combined export blob (backend stats + session timeline) over `N` days — used for download/share.    |
+| `GET /api/analytics/by-role`                                             | Backend stats grouped by agent role (WORKER/REVIEWER/ORCHESTRATOR).                                 |
+| `GET /api/analytics/by-task-type`                                        | Backend stats grouped by task type.                                                                 |
+| `GET /api/analytics/by-role-and-task-type?role=&task_type=`              | 3D stats (backend × role × task type), optionally filtered by role and/or task type query params.   |
+| `GET /api/analytics/recommend-for-task?title=&description=&role=`        | Intelligent backend selection via `BackendSelector` — combines history with task title/description. |
+
+`title` is required on `/recommend-for-task`; omitting it returns a default `claude-code` recommendation with zero confidence.
+
+______________________________________________________________________
+
 ## Troubleshooting
 
 ### Port Conflicts
diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md
@@ -46,6 +46,7 @@ max_concurrent_agents = 3
 | `require_review_approval`            | boolean        | `false`                          | Require review approval before merge                                                |
 | `serialize_merges`                   | boolean        | `true`                           | Queue merge actions                                                                 |
 | `default_agent_backend`              | string         | `"claude-code"`                  | Default worker agent                                                                |
+| `use_recommended_backend`            | boolean        | `false`                          | Auto-pick the best-performing backend per task from historical analytics (requires ≥5 prior sessions per backend × role × task-type). See [intelligent backend selection](../guides/analytics.md#intelligent-backend-selection). |
 | `additional_instructions`            | string         | `""`                             | Free-text rules appended to every agent prompt                                      |
 | `review_strictness`                  | string         | `"balanced"`                     | Review rigor. Allowed: `strict`, `balanced`, `relaxed`                              |
 | `planning_depth`                     | string         | `"always"`                       | When to create task plans. Allowed: `always`, `multi_task`, `never`                 |
diff --git a/docs/reference/mcp-tools.md b/docs/reference/mcp-tools.md
@@ -113,6 +113,36 @@ Review semantics:
 | `plugins_sync(...)`      | `destructive` | Sync issues via plugin, returns counts   |
 | `plugins_preflight(...)` | `read-only`   | Check plugin prerequisites and readiness |
 
+### Analytics tools (`toolsets/analytics.py`)
+
+All analytics tools are read-only, scoped to the active project, and return empty payloads when no project is active.
+
+| Tool                            | Annotation  | Purpose                                                             |
+| ------------------------------- | ----------- | ------------------------------------------------------------------- |
+| `analytics_backend_stats()`     | `read-only` | Per-backend session stats: count, success rate, avg duration, retry rate |
+| `analytics_session_timeline(...)` | `read-only` | Daily session counts by status over a trailing window               |
+| `analytics_export(...)`         | `read-only` | Combined backend stats + session timeline snapshot                  |
+
+#### `analytics_backend_stats`
+
+- **Arguments**: none
+- **Returns**: `{"backends": [{"agent_backend", "count", "success_rate", "avg_duration_seconds", "retry_rate"}, ...]}`
+- **Use case**: Compare agent backends across the active project to choose the most reliable one for a new task.
+
+#### `analytics_session_timeline`
+
+- **Arguments**: `days: int = 30`
+- **Returns**: `{"timeline": [{"date", "total", "completed", "failed", "cancelled", "running", "pending"}, ...]}`
+- **Use case**: Plot session volume and outcome trends over a trailing window.
+
+#### `analytics_export`
+
+- **Arguments**: `days: int = 30`
+- **Returns**: `{"exported_at", "period_days", "backend_stats", "session_timeline"}`
+- **Use case**: Snapshot aggregate analytics in a single call for archival, sharing, or offline analysis.
+
+Additional multi-dimensional analytics (by role, by task type, per-task backend recommendation) are exposed over REST at `/api/analytics/...` but are not currently wrapped as MCP tools.
+
 ______________________________________________________________________
 
 ## `task_get` API
@@ -300,7 +330,7 @@ same orchestrator-scoped MCP tool set.
 
 | Tier       | Visible tools |
 | ---------- | ------------- |
-| `readonly` | Worker-scope tools (`task_get`, `task_list`, `task_events`, `task_wait`, `run_get`, `run_cancel`, `run_detach`, `run_summary`, `review_conflicts`, `settings_get`, `plugins_preflight`, `plugins_preview`, `verify_step`, `verification_summary`, `checkpoint_create`, `checkpoint_list`, `session_rewind`, `insight_add`, `insight_list`) |
+| `readonly` | Worker-scope tools (`task_get`, `task_list`, `task_events`, `task_wait`, `run_get`, `run_cancel`, `run_detach`, `run_summary`, `review_conflicts`, `settings_get`, `plugins_preflight`, `plugins_preview`, `verify_step`, `verification_summary`, `checkpoint_create`, `checkpoint_list`, `session_rewind`, `insight_add`, `insight_list`, `analytics_backend_stats`, `analytics_session_timeline`, `analytics_export`) |
 | `default`  | Orchestrator-scope tools (worker tools plus `task_create`, `task_update`, `task_delete`, `run_start`, `review_decide`, `review_merge`, `review_rebase`, `review_verdict`, `review_clear_verdicts`, `project_list`, `project_setup`, `project_update`, `settings_set`, `audit_list`, `plugins_sync`, `persona_inspect`, `persona_import`, `persona_export`, `persona_trust`, `insight_remove`) |
 | `admin`    | Alias of `default` for MCP; currently exposes the same tool surface |
 
