Skip to content

feat: /insights command — usage analytics, cost estimation & activity patterns#552

Merged
teknium1 merged 4 commits intomainfrom
feat/insights
Mar 7, 2026
Merged

feat: /insights command — usage analytics, cost estimation & activity patterns#552
teknium1 merged 4 commits intomainfrom
feat/insights

Conversation

@teknium1
Copy link
Contributor

@teknium1 teknium1 commented Mar 6, 2026

Summary

Adds an /insights command inspired by Claude Code's /insights, adapted for Hermes Agent's multi-platform architecture. Analyzes session history from state.db to produce comprehensive usage analytics.

What it does

Generates a report covering:

  • 📋 Overview — sessions, messages, tool calls, tokens, estimated cost, active time, averages
  • 🤖 Models Used — per-model breakdown with sessions, tokens, and cost estimation
  • 📱 Platforms — CLI vs Telegram vs Discord etc. (unique to Hermes's multi-platform nature)
  • 🔧 Top Tools — most-used tools ranked with call counts and percentages
  • 📅 Activity Patterns — day-of-week bar chart, peak hours, active days, best streak
  • 🏆 Notable Sessions — longest, most messages, most tokens, most tool calls

Cost Estimation

Includes real pricing data for 25+ models (OpenAI, Anthropic, DeepSeek, Google, Meta) with fuzzy model name matching — strips provider prefixes, handles keyword heuristics (opus, sonnet, haiku, etc.), falls back to defaults for unknown models.

Three Entry Points

  1. /insights slash command in CLI — full terminal-formatted report with box-drawing and bar charts
  2. /insights slash command in gateway (Telegram/Discord/etc.) — compact markdown format
  3. hermes insights CLI subcommand — standalone usage

Configurable

  • --days N — time window (default: 30 days)
  • --source <platform> — filter by platform (cli, telegram, discord, etc.)
  • Gateway shorthand: /insights 7 sets days=7

Example Output

  ╔══════════════════════════════════════════════════════════╗
  ║                    📊 Hermes Insights                    ║
  ║                       Last 30 days                       ║
  ╚══════════════════════════════════════════════════════════╝

  📋 Overview
  ────────────────────────────────────────────────────────
  Sessions:          507           Messages:        44,517
  Tool calls:        40,841        User messages:   1,848
  ...

  🤖 Models Used
  ────────────────────────────────────────────────────────
  Model                          Sessions       Tokens     Cost
  claude-opus-4.6                     487            0 $  0.00
  ...

  📅 Activity Patterns
  ────────────────────────────────────────────────────────
  Mon  █████           59
  Tue  ███████         80
  Wed  ██████████      107
  Thu  ███████         77
  Fri  ███████████████ 155
  Sat  █               14
  Sun  █               15

Files Changed

File Purpose
agent/insights.py Core insights engine — data gathering, computation, formatting
cli.py /insights slash command handler in CLI
gateway/run.py /insights slash command handler in gateway (+ help text)
hermes_cli/commands.py Command definition for autocomplete
hermes_cli/main.py hermes insights CLI subcommand
tests/test_insights.py 56 tests — pricing, formatting, engine, edge cases

Tests

56 new tests covering:

  • Pricing lookup (exact match, prefix, heuristics, unknown, null)
  • Cost estimation calculations
  • Duration/bar chart formatting helpers
  • Empty database handling
  • Full populated database with multi-platform sessions
  • Source and days filtering
  • Terminal and gateway formatting
  • Edge cases (no tokens, no end time, no model, no tools, single platform)

Full test suite: 1982 passed, 5 skipped, 0 failures.

Future Ideas

  • LLM-powered qualitative analysis (session goal categorization, friction detection) — like Claude Code's facet extraction
  • HTML report export
  • Cost tracking over time (trend charts)
  • Per-user breakdown for multi-user gateway instances

teknium1 added 4 commits March 6, 2026 14:04
Inspired by Claude Code's /insights, adapted for Hermes Agent's multi-platform
architecture. Analyzes session history from state.db to produce comprehensive
usage insights.

Features:
- Overview stats: sessions, messages, tokens, estimated cost, active time
- Model breakdown: per-model sessions, tokens, and cost estimation
- Platform breakdown: CLI vs Telegram vs Discord etc. (unique to Hermes)
- Tool usage ranking: most-used tools with percentages
- Activity patterns: day-of-week chart, peak hours, streaks
- Notable sessions: longest, most messages, most tokens, most tool calls
- Cost estimation: real pricing data for 25+ models (OpenAI, Anthropic,
  DeepSeek, Google, Meta) with fuzzy model name matching
- Configurable time window: --days flag (default 30)
- Source filtering: --source flag to filter by platform

Three entry points:
- /insights slash command in CLI (supports --days and --source flags)
- /insights slash command in gateway (compact markdown format)
- hermes insights CLI subcommand (standalone)

Includes 56 tests covering pricing helpers, format helpers, empty DB,
populated DB with multi-platform data, filtering, formatting, and edge cases.
Comprehensive guide for using Hermes Agent with alternative LLM backends:
- Ollama (local models, zero config)
- vLLM (high-performance GPU inference)
- SGLang (RadixAttention, prefix caching)
- llama.cpp / llama-server (CPU & Metal inference)
- LiteLLM Proxy (multi-provider gateway)
- ClawRouter (cost-optimized routing with complexity scoring)
- 10+ other compatible providers table (Together, Groq, DeepSeek, etc.)
- Choosing the Right Setup decision table
- General custom endpoint setup instructions

All of these work via the existing OPENAI_BASE_URL + OPENAI_API_KEY
custom endpoint support — no code changes needed.
Custom OAI endpoints, self-hosted models, and local inference should NOT
show fabricated cost estimates. Changed default pricing from $3/$12 per
million tokens to $0/$0 for unrecognized models.

- Added _has_known_pricing() to distinguish commercial vs custom models
- Models with known pricing show $ amounts; unknown models show 'N/A'
- Overview shows asterisk + note when some models lack pricing data
- Gateway format adds '(excludes custom/self-hosted models)' note
- Added 7 new tests for custom model cost handling
…, serialization

Issues found and fixed during deep code path review:

1. CRITICAL: Prefix matching returned wrong prices for dated model names
   - 'gpt-4o-mini-2024-07-18' matched gpt-4o ($2.50) instead of gpt-4o-mini ($0.15)
   - Same for o3-mini→o3 (9x), gpt-4.1-mini→gpt-4.1 (5x), gpt-4.1-nano→gpt-4.1 (20x)
   - Fix: use longest-match-wins strategy instead of first-match
   - Removed dangerous key.startswith(bare) reverse matching

2. CRITICAL: Top Tools section was empty for CLI sessions
   - run_agent.py doesn't set tool_name on tool response messages (pre-existing)
   - Insights now also extracts tool names from tool_calls JSON on assistant
     messages, which IS populated for all sessions
   - Uses max() merge strategy to avoid double-counting between sources

3. SELECT * replaced with explicit column list
   - Skips system_prompt and model_config blobs (can be thousands of chars)
   - Reduces memory and I/O for large session counts

4. Sets in overview dict converted to sorted lists
   - models_with_pricing / models_without_pricing were Python sets
   - Sets aren't JSON-serializable — would crash json.dumps()

5. Negative duration guard
   - end > start check prevents negative durations from clock drift

6. Model breakdown sort fallback
   - When all tokens are 0, now sorts by session count instead of arbitrary order

7. Removed unused timedelta import

Added 6 new tests: dated model pricing (4), tool_calls JSON extraction,
JSON serialization safety. Total: 69 tests.
@teknium1 teknium1 merged commit 6d38047 into main Mar 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant