feat: add Hermes token usage source - #88
Conversation
|
Follow-up verification completed on commit
The follow-up only resolves test-target Clippy warnings; |
|
@nicosuave This is ready for maintainer review. The implementation and follow-up all-target Clippy cleanup are pushed, and the full local verification results are recorded above. |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b943b45f0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed all four Codex review findings in commit Verification after the final repair:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c33ec61d2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed the three latest Codex findings in commit Additional hardening included:
Final local verification:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 36981df8f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Thanks for this @MaterAnime |
…eat/hermes-usage-upstream # Conflicts: # src/sources/hermes.rs # src/usage.rs
|
Merged Nico’s concurrent fixes with the independently verified aggregate-accounting repair on head The merged result addresses all four latest findings and includes:
Merged-tree verification:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 84fa162d1d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Fixed the non-UTF-8 WAL path finding on head Verification:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 366bd8f03a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| model: Option<String>, | ||
| tokens: TokenBuckets, | ||
| source_cost_usd: Option<f64>, | ||
| cost_authoritative: bool, |
There was a problem hiding this comment.
When upgrading with a populated usage_file_cache, inserting this field changes the positional postcard layout while the usage parser versions for Claude, Codex, Cursor, OpenCode, Pi, OpenClaw, and Copilot remain unchanged. Their rows are therefore selected as cache hits, but decoding encounters the old dedupe_confidence string where this Boolean is expected and reparses every cached history file, making the first usage or TUI scan unnecessarily behave like a cold scan for large histories. Decode the previous event format or introduce a cache-format migration that supplies false for the new internal fields.
Useful? React with 👍 / 👎.
|
|
||
| fn split_roots(roots: &std::ffi::OsStr) -> Vec<PathBuf> { | ||
| roots | ||
| .to_string_lossy() |
There was a problem hiding this comment.
On Unix, when HERMES_PROFILE_ROOTS contains a path component with non-UTF-8 bytes, this lossy conversion replaces those bytes before discovery, so the resulting PathBuf does not identify the configured directory and its state.db is never scanned. Fresh evidence beyond the prior WAL-sidecar issue is that the explicit-root path is still converted before discovery; split the OsStr using native bytes so the WAL fix is reachable for this configuration.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 366bd8f03a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| .into_iter() | ||
| .map(|file| file.path) | ||
| .collect::<Vec<_>>(); | ||
| scan_files_cached( |
There was a problem hiding this comment.
When default Hermes profile entries differ only in invalid UTF-8 bytes and their databases have matching size/mtime, this scanner passes both native paths to scan_files_cached, but that function persists and looks them up using path.to_string_lossy() under PRIMARY KEY (source, path). Both paths collapse to the same key, so one cache row overwrites the other and a subsequent scan can reuse one profile's events for both, double-counting it while omitting the other. Fresh evidence beyond the prior WAL and explicit-root findings is that the primary usage cache key remains lossy; preserve native path identity there as well.
Useful? React with 👍 / 👎.
| let inconsistent = current != raw_current; | ||
| invalid_model |= inconsistent; | ||
| summed = add(summed, ¤t); | ||
| if !is_zero(¤t) { |
There was a problem hiding this comment.
When model-detail rows exceed a session aggregate and a later row is capped entirely to zero, this guard suppresses the only event carrying inconsistent = true; earlier retained model events remain non-conservative, and without an authoritative session cost there is no residual event either. For example, two 10-token models against a 10-token aggregate produce only the lexicographically first model with conservative_events == 0, silently presenting an arbitrary model/task attribution as reliable. Propagate invalid_model to the retained session events or emit an explicit conservative fallback.
Useful? React with 👍 / 👎.
|
The Codex toil is getting to be a little too much; feel free to open a followup PR if you notice anything. |
2 participants
Summary
Adds Hermes Agent as a first-class Memex token-usage source.
state.dband immediate profile databasessessionsaggregates and currentsession_model_usagemodel/task rowsPrivacy and safety
The adapter reads usage counters and limited attribution metadata only. It does not query message text, system prompts, tool/reasoning content, credentials, auth, memory, skills, plugins, or cron data. JSON/JSONL transcripts are not treated as Hermes usage sources.
SQLite databases are opened read-only. WAL metadata participates in cache freshness; SHM coordination metadata does not. Parser version 4 invalidates older Hermes projections, and malformed legacy dependency cache entries fail closed by reparsing.
Accounting model
Hermes stores disjoint buckets:
total = input_tokens + cache_read_tokens + cache_write_tokens + output_tokensFor databases with
session_model_usage, per-model/task rows are emitted first and capped conservatively against the session aggregate. Any positive residual is retained once at session level. Legacy databases use the session aggregate directly.Verification
cargo fmt --checkcargo clippy -- -D warningscargo test: 301 passed, 0 failed, 2 ignoredThe upstream branch intentionally excludes the local installer/rollback scripts and Herdr cockpit metadata used during development.