Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# memex

Fast local history search for Claude, Codex CLI, Cursor, OpenCode, Pi Coding Agent, OpenClaw, and GitHub Copilot CLI logs. Uses BM-25 and optionally embeds your transcripts locally for hybrid search.
Fast local history search for Claude, Codex CLI, Cursor, OpenCode, Pi Coding Agent, OpenClaw, GitHub Copilot CLI, and Hermes usage records. Uses BM-25 and optionally embeds your transcripts locally for hybrid search.

Mostly intended for agents to use via skill. The intended workflow is to ask agent about a previous session & then the agent can narrow things down & retrieve history as needed.

Expand Down Expand Up @@ -210,19 +210,20 @@ Token tracking is disabled by default because it scans and caches local agent lo
token_usage = true
```

Then reconstruct historical token usage from local Claude Code, Codex, Cursor, OpenCode, Pi, OpenClaw, and Copilot logs:
Then reconstruct historical token usage from local Claude Code, Codex, Cursor, OpenCode, Pi, OpenClaw, Copilot, and Hermes records:

```
memex usage
memex usage --source codex --since 2026-07-01
memex usage --source hermes --since 2026-07-01
memex usage --json --events
```

`--cost auto` prefers a provider-stored request cost and otherwise applies the versioned built-in API price catalog. `--cost source` uses only stored costs; `--cost reprice` always applies the catalog. Calculated costs are API-equivalent estimates, not subscription charges. Events with unknown models or prices remain in token totals and are reported as unpriced.

Each source also reports prompt-cache efficiency: the cache hit rate, plus an estimate of cache waste — prompt tokens that were in the previous request's prompt but were re-billed at input rates instead of read from cache, priced at catalog rates and attributed to idle gaps past the cache TTL or model switches where those apply. Waste is estimated per transcript file chain and errs toward undercounting: subagent sidechains, ambiguous dedupe deltas, and prompts that shrink past compaction are not counted.

Local token history is reconstructed usage. It is deliberately kept separate from authoritative subscription quota percentages and reset windows.
Local token history is reconstructed usage. It is deliberately kept separate from authoritative subscription quota percentages and reset windows. Hermes usage is read from `state.db` in the Hermes root and immediate profile directories (`HERMES_PROFILE_ROOTS`, `HERMES_HOME`, or `HERMES_STATE_DIR`, with safe local defaults), opened read-only and WAL-compatible. The `sessions` aggregate is used for legacy databases; newer `session_model_usage` delta rows are emitted by model/task and reconciled against the session aggregate so historical seeded rows count once and positive residuals are retained. Snapshots, backups, arbitrary nested databases, JSON/JSONL transcripts, and auth, config, memory, skills, plugins, and cron paths are excluded. Hermes queries never read message, system-prompt, tool, reasoning-text, or credential tables. Usage output contains counters and metadata only. Any API-equivalent cost estimate is analytical and is not a Hermes subscription quota measurement; source-stored API costs are not quota percentages. Hermes parser-version changes invalidate only Hermes usage cache rows.

When token tracking is enabled, press `Ctrl+T` on the TUI home screen to toggle the 30-day activity chart between session count and token volume. Token activity is loaded lazily and cached when first shown.

Expand Down Expand Up @@ -266,7 +267,7 @@ This detects which tools are installed (Claude/Codex/OpenCode/Pi) and presents a
- `--role <user|assistant|tool_use|tool_result>`
- `--tool <tool_name>`
- `--session <session_id>`
- `--source claude|codex|cursor|opencode|pi|openclaw|copilot`
- `--source claude|codex|cursor|opencode|pi|openclaw|copilot|hermes`
- `--since <iso|unix>` / `--until <iso|unix>`
- `--limit <n>`
- `--min-score <float>`
Expand Down
11 changes: 6 additions & 5 deletions src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ use std::time::Duration;
#[command(
name = "memex",
version,
about = "Fast local history search for Claude, Codex, Cursor, OpenCode, Pi, OpenClaw, and Copilot",
about = "Fast local history search for Claude, Codex, Cursor, OpenCode, Pi, OpenClaw, Copilot, and Hermes",
after_help = "\
QUICK START:
memex # Browse sessions interactively
Expand Down Expand Up @@ -183,7 +183,7 @@ OUTPUT FIELDS (--fields):
/// Filter by session ID
#[arg(long)]
session: Option<String>,
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, or copilot
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, copilot, or hermes
#[arg(long)]
source: Option<SourceFilter>,
/// Use semantic (embedding-based) search instead of keyword search
Expand Down Expand Up @@ -301,7 +301,7 @@ EXAMPLES:
/// Filter by project (repository grouping)
#[arg(long)]
project: Option<String>,
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, or copilot
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, copilot, or hermes
#[arg(long)]
source: Option<SourceFilter>,
/// Only include sessions active on or after this date/timestamp
Expand Down Expand Up @@ -336,7 +336,7 @@ EXAMPLES:
memex usage --source codex --since 2026-07-01
memex usage --json")]
Usage {
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, or copilot
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, copilot, or hermes
#[arg(long)]
source: Option<SourceFilter>,
/// Only include events on or after this date/timestamp
Expand Down Expand Up @@ -452,7 +452,7 @@ enum HerdrCommand {
/// Prefer sessions from this directory (falls back to the global latest)
#[arg(long)]
cwd: Option<PathBuf>,
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, or copilot
/// Filter by source: claude, codex, cursor, opencode, pi, openclaw, copilot, or hermes
#[arg(long)]
source: Option<SourceFilter>,
/// Path to memex data directory [default: ~/.memex]
Expand Down Expand Up @@ -2544,6 +2544,7 @@ fn run_share(session_id: String, title: Option<String>, root: Option<PathBuf>) -
crate::types::SourceKind::Pi => "pi",
crate::types::SourceKind::OpenClaw => "openclaw",
crate::types::SourceKind::Copilot => "copilot",
crate::types::SourceKind::Hermes => "hermes",
};
let source_path = &record.source_path;

Expand Down
5 changes: 3 additions & 2 deletions src/ingest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -743,6 +743,7 @@ pub fn ingest_all(
SourceKind::Copilot => {
parse_copilot_session(task, &tx_record, &tx_update, &next_doc_id, &progress)
}
SourceKind::Hermes => Err(anyhow!("Hermes indexing is not supported")),
};
finish_file_task(task, &progress, &parse_skipped, result)
})
Expand Down Expand Up @@ -2926,8 +2927,8 @@ mod tests {
let (tx_update, rx_update) = unbounded();
let next_doc_id = AtomicU64::new(1);
let progress = Arc::new(Progress::new(
[0, 0, 0, 0, 0, 0, meta.len()],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, meta.len(), 0],
[0, 0, 0, 0, 0, 0, 1, 0],
false,
));

Expand Down
1 change: 1 addition & 0 deletions src/progress.rs
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ fn progress_label(source: SourceKind) -> &'static str {
SourceKind::Pi => "pi",
SourceKind::OpenClaw => "openclaw",
SourceKind::Copilot => "copilot",
SourceKind::Hermes => "hermes",
}
}

Expand Down
1 change: 1 addition & 0 deletions src/resume.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ pub fn resume_template(config: &UserConfig, source: SourceKind, remote: bool) ->
SourceKind::Pi => config.pi_resume_cmd.clone(),
SourceKind::OpenClaw => return None,
SourceKind::Copilot => config.copilot_resume_cmd.clone(),
SourceKind::Hermes => None,
};
configured.or_else(|| default_resume_template(source.label(), remote))
}
Expand Down
18 changes: 18 additions & 0 deletions src/sources/audit.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,13 @@ pub fn audit_installed_sources(source: Option<SourceFilter>) -> Result<Vec<Sourc
.map(|file| file.path)
.collect(),
);
push(
SourceKind::Hermes,
super::hermes::discover()
.into_iter()
.map(|file| file.path)
.collect(),
);

groups
.into_iter()
Expand Down Expand Up @@ -100,6 +107,12 @@ pub fn audit_files(source: SourceKind, files: &[PathBuf]) -> Result<SourceAudit>
}

fn audit_file(source: SourceKind, path: &Path, audit: &mut SourceAudit) -> Result<()> {
if source == SourceKind::Hermes {
// Hermes usage truth is SQLite aggregate data. Audit must not reinterpret the
// database as JSON, and in particular must not read transcript/message columns.
std::fs::File::open(path)?;
return Ok(());
}
let reader = std::io::BufReader::new(std::fs::File::open(path)?);
for line in reader.lines() {
let line = line?;
Expand Down Expand Up @@ -191,6 +204,11 @@ fn record_semantics(source: SourceKind, value: &Value, top_level: &str, audit: &
}
record_content_blocks(value.get("content"), audit);
}
SourceKind::Hermes => {
if value.get("records").and_then(Value::as_array).is_some() {
increment(&mut audit.semantic_types, "records");
}
}
}
}

Expand Down
2 changes: 2 additions & 0 deletions src/sources/claude.rs
Original file line number Diff line number Diff line change
Expand Up @@ -628,8 +628,10 @@ pub(crate) fn parse_usage_file(path: &Path) -> Result<Vec<UsageEvent>> {
.map(str::to_string),
tokens,
source_cost_usd: value.get("costUSD").and_then(|value| value.as_f64()),
cost_authoritative: false,
dedupe_confidence: if exact_dedupe { "exact" } else { "heuristic" },
conservative_undercount: false,
cache_chain_excluded: false,
sidechain: value
.get("isSidechain")
.and_then(|value| value.as_bool())
Expand Down
4 changes: 4 additions & 0 deletions src/sources/codex.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1236,9 +1236,11 @@ pub(crate) fn parse_usage_file(
delta.reasoning,
),
source_cost_usd: None,
cost_authoritative: false,
dedupe_confidence: "strong",
conservative_undercount: counter.interleaved
|| (parent.is_some() && !fork_resolved),
cache_chain_excluded: false,
sidechain: false,
source_order,
});
Expand Down Expand Up @@ -1281,8 +1283,10 @@ pub(crate) fn parse_usage_file(
tokens.reasoning,
),
source_cost_usd: None,
cost_authoritative: false,
dedupe_confidence: "strong",
conservative_undercount: false,
cache_chain_excluded: false,
sidechain: false,
source_order,
});
Expand Down
2 changes: 2 additions & 0 deletions src/sources/copilot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -680,8 +680,10 @@ fn extract_usage(
.max(attribute_u64(attributes, "gen_ai.usage.reasoning_tokens")),
),
source_cost_usd: None,
cost_authoritative: false,
dedupe_confidence: "exact",
conservative_undercount: false,
cache_chain_excluded: false,
sidechain: false,
source_order: index,
});
Expand Down
2 changes: 2 additions & 0 deletions src/sources/cursor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -488,12 +488,14 @@ fn extract_usage(
}),
tokens: TokenBuckets::disjoint(input, 0, 0, output),
source_cost_usd: None,
cost_authoritative: false,
dedupe_confidence: if table == "cursorDiskKV" {
"exact"
} else {
"strong"
},
conservative_undercount: false,
cache_chain_excluded: false,
sidechain: false,
source_order: 0,
});
Expand Down
Loading