Skip to content

feat(usage): live token burn and spend for running agents - #141

Merged
murdore merged 1 commit into
releasefrom
feat/session-usage-monitor
Aug 14, 2026
Merged

feat(usage): live token burn and spend for running agents#141
murdore merged 1 commit into
releasefrom
feat/session-usage-monitor

Conversation

@murdore

@murdore murdore commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Adds a Burn panel to the dashboard answering what the agents are consuming right now, per session, read from Claude Code transcripts. Backed by a new GET /api/usage.

Why the counting is not trivial

Deduplication, and which copy to keep. Claude Code splits one API response across several JSONL entries sharing a message.id, each carrying its own usage block.

approach result on a real 11.5k-line session
sum every entry output tokens +124.6% — more than double
keep the first entry per id -4.3% — undercounts
keep the last entry per id correct

Usage accumulates as the response streams: 575 of 6681 responses disagreed across their entries, and in 575 of 575 the last entry was the larger one. This matches what docs/CLAUDE-CODE-SESSION-JSONL-SCHEMA.md already documents — "the last entry has the final/accurate usage". requestId is not a usable key; it appeared on 1 of 4144 records.

Scale. The transcript corpus measured 9.3 GB across 15,244 files, 1.5 GB of it touched in a single day. 15,155 of those files are subagent transcripts nested under <session>/subagents/, which a flat readdir misses entirely — that is the large majority of agent spend. Reading files whole took 21s, useless for an endpoint a phone polls.

Transcripts are append-only and chronological, so everything inside a trailing window sits at the end of the file. Only the last 4 MB of files whose mtime falls inside the window are read:

cold warm
snapshot, 60-minute window 3.2s 130ms

This is why the endpoint reports a window rather than all time. That is the honest shape for a live monitor, and it is documented as such.

Cost is never guessed

Transcripts record tokens but no cost, so spend must be computed from a rate table — which makes a missing model a correctness hazard, since it would silently contribute 0 to a total that still looks authoritative.

  • Rates come from SHOOTER_MODEL_PRICING or ~/.shooter/pricing.json, in USD per million tokens.
  • A model with no configured rate reports exact token counts and renders cost as an em dash, never $0.00. priced: false propagates from model to session to snapshot — unknown is contagious.
  • The built-in table covers only models whose public list pricing is long settled. Newer ones are deliberately absent: the operator knows what they actually pay.
  • truncatedFiles surfaces when even the tail budget was short, so figures are never quietly incomplete.

Scope

Delivered: per-session token totals, burn rate (tokens/min, $/hr), per-model rollup, cache-hit share, 15m/1h/6h/24h windows.

Not included: CPU load (not present in transcripts — needs separate process sampling) and a "one-shot rate" (no definition of a shot exists in the data). Cache-hit share stands in as the efficiency signal, since cache reads are roughly an order of magnitude cheaper and the ratio is directly derivable.

Verification

  • Full suite green, including 15 new checks in tests/usage-reader.test.cjs (wired into pnpm test)
  • pnpm run check — 0 errors, 0 warnings across 1015 files; eslint and prettier clean
  • Production build succeeds; server run live — 401 unauthenticated, correct payload authenticated, priced path confirmed
  • Types generated from specs/types/usage.yaml via pnpm gen:types; the two internal shapes carrying a Map live in src/lib/types/usage.ts per the type-governance rules

Docs

docs/API-REFERENCE.md gains the GET /api/usage reference; docs/ENVIRONMENT.md gains SHOOTER_MODEL_PRICING and SHOOTER_CLAUDE_PROJECTS_DIR.

Summary by CodeRabbit

  • New Features

    • Added a dashboard usage panel showing token consumption, costs, burn rate, response counts, pricing warnings, and per-session details.
    • Added an authenticated usage API with configurable time windows and result limits.
    • Added model pricing support, including custom pricing overrides and unpriced-model indicators.
  • Documentation

    • Documented the usage API and new environment variables.
    • Added public schemas for usage summaries and pricing data.
  • Tests

    • Added coverage for usage aggregation, pricing, filtering, deduplication, and error handling.

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown

Single Commit Policy - COMPLIANT

Status: Policy requirements met - 1 commit - Valid format - Ready for merge

View validation details

Commit Details

  • Hash: 6202cf9f203fef11f3bf800e3077c90a07f5a51c
  • Message: feat(usage): live token burn and spend for running agents
  • Author: Sachin Sharma

Validation Results

  • Single commit requirement met
  • No merge commits in branch
  • Semantic commit message format verified
  • Ready for squash merge to release branch

Automated validation by Shooter Single Commit Enforcement

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@murdore, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 103 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8d93ba50-304b-4b8b-9c4e-68b8a4668205

📥 Commits

Reviewing files that changed from the base of the PR and between 5d07c34 and 6202cf9.

⛔ Files ignored due to path filters (1)
  • src/lib/types/generated/Usage.ts is excluded by !**/generated/**
📒 Files selected for processing (6)
  • specs/types/usage.yaml
  • src/lib/modules/client/dashboard/UsagePanel.svelte
  • src/lib/modules/server/sessions/usage-pricing.ts
  • src/lib/modules/server/sessions/usage-reader.ts
  • src/routes/api/usage/+server.ts
  • tests/usage-reader.test.cjs

Walkthrough

Added end-to-end usage reporting. The server reads Claude transcripts, applies model pricing, aggregates usage, exposes /api/usage, and renders a polling dashboard panel. Schemas, environment documentation, and comprehensive tests were added.

Changes

Usage reporting

Layer / File(s) Summary
Usage contracts and pricing
specs/types/usage.yaml, specs/types/index.yaml, src/lib/types/usage.ts, src/lib/types/index.ts, src/lib/modules/server/sessions/usage-pricing.ts
Added usage schemas, internal aggregation types, configurable model pricing, longest-prefix matching, cost calculation, and pricing cache reset support.
Transcript discovery and aggregation
src/lib/modules/server/sessions/usage-reader.ts
Added recursive transcript discovery, bounded reads, time-window filtering, response deduplication, token and cost rollups, burn-rate calculations, sorting, limits, and truncation reporting.
Authenticated usage API
src/routes/api/usage/+server.ts, docs/API-REFERENCE.md, docs/ENVIRONMENT.md
Added the authenticated GET /api/usage endpoint with bounded parameters, refresh handling, five-second caching, and API documentation.
Dashboard usage panel
src/lib/modules/client/dashboard/UsagePanel.svelte, src/lib/modules/client/dashboard/index.ts, src/routes/+page.svelte
Added the polling usage panel and placed it before AutopilotPanel in the dashboard.
Usage behavior validation
tests/usage-reader.test.cjs, package.json
Added tests for pricing, aggregation, deduplication, windows, burn rates, discovery, malformed input, and synthetic models. Included the tests in the package test script.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: 🟡 Moderate · up to 5d07c

This PR adds live usage reporting, but the current implementation can silently undercount usage for oversized transcript records and can block server request handling during cold scans; stale window responses and invalid pricing inputs also create bounded correctness risks. These issues should be fixed or explicitly accepted before merge.

Sequence Diagram(s)

sequenceDiagram
  participant Dashboard
  participant UsageAPI
  participant UsageReader
  participant ClaudeTranscripts
  Dashboard->>UsageAPI: GET /api/usage with window and limit
  UsageAPI->>UsageReader: Request usageSnapshot
  UsageReader->>ClaudeTranscripts: Discover and read transcript tails
  ClaudeTranscripts-->>UsageReader: Transcript records
  UsageReader-->>UsageAPI: Aggregated usage snapshot
  UsageAPI-->>Dashboard: JSON usage response
Loading

Poem

I’m a rabbit with tokens to count,
Through transcript trails, the totals mount.
Prices bloom, caches read,
Burn rates hop ahead,
And the dashboard shows what you spent.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: live token burn and spend reporting for running agents.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/session-usage-monitor

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@specs/types/usage.yaml`:
- Around line 215-224: Align the UsageSnapshot.sessions contract with the
implementation by updating its description near the sessions array to state that
entries are ordered by largest token count first, matching the
tokens.totalTokens sort in usage-reader.ts and UsagePanel.svelte’s
busiest-session behavior.

In `@src/lib/modules/client/dashboard/UsagePanel.svelte`:
- Around line 77-99: Update refresh and selectWindow to track the latest
request, using a request sequence or AbortController. Only the current request
may update snapshot, failed, or loading, including success, failure, catch, and
finally paths, so an older window response cannot overwrite the selected
window’s state.

In `@src/lib/modules/server/sessions/usage-pricing.ts`:
- Around line 145-171: Update the validation in the rate-parsing loop that
populates out to skip blank or whitespace-only model keys and reject any
supplied negative input, output, cacheRead, or cacheWrite rates. Preserve
fallback handling for absent cache rates, and only add entries to out when the
model ID is nonblank and all provided rates are nonnegative.

In `@src/lib/modules/server/sessions/usage-reader.ts`:
- Around line 438-446: Update readTail to explicitly report that the leading
record is incomplete when the capped tail contains no newline, rather than
returning the partial JSON unchanged. Propagate this metadata through the
transcript-reading flow and set truncated/truncatedFiles when the capped tail
has no complete record, while preserving normal parsing for tails containing
complete newline-delimited records.
- Around line 105-129: Convert usageSnapshot and its transcript
discovery/parsing path to use asynchronous fs.promises operations instead of
synchronous traversal and readTail calls, including awaiting any async helpers
and preserving the existing aggregation results. Update the /api/usage handler
to await usageSnapshot, while keeping the current cache behavior after the
asynchronous boundary.

In `@tests/usage-reader.test.cjs`:
- Around line 46-55: Update the shared check function to clean and recreate
CORPUS in a finally block so cleanup runs after both successful and failed
assertions. Reset the transcript cache in that same finally block, preserving
the existing pass/fail reporting and process exit behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bc6cb4d1-7f00-447d-a198-a8f84e509fa6

📥 Commits

Reviewing files that changed from the base of the PR and between 0503b38 and 5d07c34.

⛔ Files ignored due to path filters (2)
  • src/lib/types/generated/Usage.ts is excluded by !**/generated/**
  • src/lib/types/generated/index.ts is excluded by !**/generated/**
📒 Files selected for processing (14)
  • docs/API-REFERENCE.md
  • docs/ENVIRONMENT.md
  • package.json
  • specs/types/index.yaml
  • specs/types/usage.yaml
  • src/lib/modules/client/dashboard/UsagePanel.svelte
  • src/lib/modules/client/dashboard/index.ts
  • src/lib/modules/server/sessions/usage-pricing.ts
  • src/lib/modules/server/sessions/usage-reader.ts
  • src/lib/types/index.ts
  • src/lib/types/usage.ts
  • src/routes/+page.svelte
  • src/routes/api/usage/+server.ts
  • tests/usage-reader.test.cjs

Comment thread specs/types/usage.yaml
Comment thread src/lib/modules/client/dashboard/UsagePanel.svelte
Comment on lines +145 to +171
const out: Record<string, ModelRate> = {};
for (const [model, value] of Object.entries(raw as Record<string, unknown>)) {
if (typeof value !== 'object' || value === null) {
console.warn(`[usage] ignoring rate for "${model}" in ${source}: not an object`);
continue;
}
const v = value as Record<string, unknown>;
const num = (key: string): number => (typeof v[key] === 'number' ? v[key] : NaN);
const input = num('input');
const output = num('output');
// A rate missing input/output cannot price anything; cache rates may
// legitimately be absent (older models had no prompt cache) and fall back
// to the uncached rates rather than to zero, which would under-report.
if (!Number.isFinite(input) || !Number.isFinite(output)) {
console.warn(
`[usage] ignoring rate for "${model}" in ${source}: input/output must be numbers`
);
continue;
}
const cacheWrite = num('cacheWrite');
const cacheRead = num('cacheRead');
out[model] = {
cacheRead: Number.isFinite(cacheRead) ? cacheRead : input,
cacheWrite: Number.isFinite(cacheWrite) ? cacheWrite : input,
input,
output,
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject empty model keys and negative rates.

An empty model key matches every model in rateFor because every string starts with ''. Negative finite values also pass validation and produce negative costUsd. Reject blank model IDs and require every supplied rate to be greater than or equal to zero before adding it to out.

Proposed validation
   for (const [model, value] of Object.entries(raw as Record<string, unknown>)) {
+    if (!model.trim()) {
+      console.warn(`[usage] ignoring blank model id in ${source}`);
+      continue;
+    }
     if (typeof value !== 'object' || value === null) {
       console.warn(`[usage] ignoring rate for "${model}" in ${source}: not an object`);
       continue;
@@
-    if (!Number.isFinite(input) || !Number.isFinite(output)) {
+    if (!Number.isFinite(input) || input < 0 || !Number.isFinite(output) || output < 0) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/modules/server/sessions/usage-pricing.ts` around lines 145 - 171,
Update the validation in the rate-parsing loop that populates out to skip blank
or whitespace-only model keys and reject any supplied negative input, output,
cacheRead, or cacheWrite rates. Preserve fallback handling for absent cache
rates, and only add entries to out when the model ID is nonblank and all
provided rates are nonnegative.

Comment thread src/lib/modules/server/sessions/usage-reader.ts Outdated
Comment thread src/lib/modules/server/sessions/usage-reader.ts Outdated
Comment thread tests/usage-reader.test.cjs

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Reviewed the 16 changed files for PR #141. The feature is well-documented and the test coverage is thoughtful, but there are correctness and availability issues that need to be addressed before merging.

New issues raised

Severity Count Files
🔒 CRITICAL 1 usage-reader.ts
⚠️ MAJOR 4 usage-reader.ts, usage-pricing.ts, +server.ts, UsagePanel.svelte
💡 MINOR 2 usage-reader.test.cjs, usage.yaml

Blocking concerns

  1. Synchronous bulk I/O on a public endpoint (usage-reader.ts)usageSnapshot walks directories and reads up to 8 GB of transcript data synchronously. Called directly from /api/usage, this blocks the Node event loop and stalls every concurrent session/WebSocket on a host exposed via Cloudflare Tunnel. Convert the reader to fs.promises and await it from the route.

  2. Module-level response cache race (src/routes/api/usage/+server.ts)cached, cachedKey, and cachedAt are shared across all requests. Concurrent requests for different windows can overwrite each other, returning the wrong snapshot for the requested window.

  3. Stale request overwriting selected window (UsagePanel.svelte) — Rapid window switches can leave the UI showing data for a different window than the active button.

  4. Invalid rate-table entries accepted (usage-pricing.ts) — Empty model keys match every model id via prefix, and negative rates produce negative costs. Both corrupt spend reporting.

  5. Large single-record transcripts silently omitted (usage-reader.ts) — When a capped tail contains no newline, the partial record is dropped but truncatedFiles is not incremented.

Non-blocking

  • Test cleanup should run in a finally block so failures do not leak state between checks.
  • UsageSnapshot.sessions description should match the implementation (sorted by token count, not recency).

Note on existing comments

Several CodeRabbit review threads raised overlapping points. I have not duplicated them; instead I replied to the still-unresolved ones with concrete fixes. Please resolve those threads as part of the revision.

Next steps

  1. Make usageSnapshot/listTranscripts/readTail asynchronous and await them in +server.ts.
  2. Remove or correctly key the module-level cache in +server.ts.
  3. Add request sequencing to UsagePanel.svelte.
  4. Reject blank model ids and negative rates in usage-pricing.ts.
  5. Report capped tails with no complete record as truncated in usage-reader.ts.
  6. Clean up tests in a finally block and align the YAML session-order description.

Once these are addressed, happy to re-review.

* burn rate always cover every scanned transcript, so the headline figures
* never silently describe a subset of the window.
*/
export function usageSnapshot(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 CRITICAL: Synchronous I/O blocks the event loop on a public endpoint.

usageSnapshot performs recursive directory walks and reads up to MAX_FILES × TAIL_BUDGET_BYTES (8 GB) synchronously. Because /api/usage calls this directly, a single cold request can freeze the Node server for multiple seconds. With the Cloudflare Tunnel exposing this host, that stalls all concurrent sessions, WebSockets, and terminals.

The CodeRabbit thread raised the same concern; it is unresolved. Convert the reader to fs.promises and await it from the route, or move aggregation to a background worker/cache so the request path never performs synchronous bulk I/O.

Suggested direction:

export async function usageSnapshot(...): Promise<UsageSnapshot> {
  const candidates = (await listTranscriptsAsync())
    .filter(...)
    .sort(...)
    .slice(0, MAX_FILES);
  for (const t of candidates) { ... await refreshAsync(t) ... await parseTranscriptAsync(...) }
}

const text = buffer.toString('utf8', 0, read);
// The first line is almost certainly cut mid-record; drop it rather than
// feed a truncated JSON fragment to the parser.
return { complete: false, text: text.slice(text.indexOf('\n') + 1) };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ MAJOR: Truncated single-record transcripts are not flagged as truncated.

When size > TAIL_BUDGET_BYTES and the tail contains no newline, text.indexOf('\n') returns -1, so text.slice(0) returns the partial record unchanged. parseTranscript then fails to parse it, result.requests stays zero, and truncated remains false. A usage-bearing JSONL record larger than 4 MiB is silently omitted without incrementing truncatedFiles.

This is the same issue the existing CodeRabbit thread identified; it is still unresolved. Return explicit incomplete-leading-record metadata from readTail and mark the transcript truncated when no complete record survives the cap.

Suggested fix:

const firstNewline = text.indexOf('\n');
if (firstNewline === -1) {
  return { complete: false, hasCompleteRecord: false, text: '' };
}
return { complete: false, hasCompleteRecord: true, text: text.slice(firstNewline + 1) };

and in parseTranscript set result.truncated = !complete && !hasCompleteRecord || (!complete && oldestSeenMs >= cutoffMs && result.requests > 0);

cacheWrite: Number.isFinite(cacheWrite) ? cacheWrite : input,
input,
output,
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ MAJOR: Empty model keys and negative rates are accepted.

An empty model key matches every model id in rateFor because every string starts with ''. Negative finite rates also pass validation and produce negative costUsd. Both corrupt the spend calculation and the unpricedModels signal.

This is the same issue the existing CodeRabbit thread raised; it remains unresolved. Reject blank model IDs and require every supplied rate to be ≥ 0.

Suggested fix:

for (const [model, value] of Object.entries(raw as Record<string, unknown>)) {
  if (!model.trim()) {
    console.warn(`[usage] ignoring blank model id in ${source}`);
    continue;
  }
  ...
  if (!Number.isFinite(input) || input < 0 || !Number.isFinite(output) || output < 0) {
    console.warn(`[usage] ignoring rate for "${model}" in ${source}: input/output must be non-negative numbers`);
    continue;
  }
  const cacheWrite = num('cacheWrite');
  const cacheRead = num('cacheRead');
  if (Number.isFinite(cacheWrite) && cacheWrite < 0) { ... }
  if (Number.isFinite(cacheRead) && cacheRead < 0) { ... }
}

Comment thread src/routes/api/usage/+server.ts Outdated
cached = null;
}
if (!cached || now - cachedAt >= CACHE_TTL_MS) {
cached = usageSnapshot({ sessionLimit, windowMinutes });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ MAJOR: Module-level mutable cache is shared across concurrent requests and callers.

cached, cachedKey, and cachedAt are top-level let variables. In SvelteKit/Node they are shared across all requests in the same process. This creates two problems:

  1. Race: request A for window=15 starts computing; request B for window=60 arrives, sees a stale/missing cache, computes; whichever finishes last overwrites cached with the wrong key, so subsequent requests may return a snapshot for a different window than their key.
  2. Cross-tenant leakage: although auth happens first, the cache itself is not keyed by caller identity. In this repo the API_KEY is a single shared secret, so this is mainly a correctness issue, but it also means any refresh=true caller invalidates the cache for everyone.

Keep the cache inside the request handler (cheap enough given the 5 s TTL and that the heavy work dominates), or guard the module cache with the exact key match and never let a different key overwrite it.

Suggested fix:

export const GET: RequestHandler = ({ request, url }) => {
  const authError = validateAuth(request);
  if (authError) return authError;

  const windowMinutes = intParam(url.searchParams.get('window'), 60, 1, 1440);
  const sessionLimit = intParam(url.searchParams.get('limit'), 20, 1, 200);

  const snapshot = usageSnapshot({ sessionLimit, windowMinutes });
  return json(snapshot);
};

If caching is retained, store it per-request-key and never allow a mismatched key to overwrite.

windowMinutes = minutes;
loading = true;
void refresh();
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ MAJOR: Stale responses can overwrite the currently-selected window.

refresh() has no request sequencing. If the user clicks 24h and then quickly 15m, the slower 24h request can land after the 15m response and overwrite snapshot. The UI then shows 24-hour totals while the buttons highlight 15m.

This is the same issue the existing CodeRabbit thread raised; it remains unresolved. Track the latest request and ignore older responses.

Suggested fix:

let requestSeq = 0;

async function refresh(): Promise<void> {
  const seq = ++requestSeq;
  try {
    const res = await fetch(`/api/usage?window=${windowMinutes}&limit=40`, {
      headers: authHeaders(),
    });
    if (seq !== requestSeq) return;
    if (res.ok) {
      snapshot = (await res.json()) as UsageSnapshot;
      failed = false;
    } else {
      failed = true;
    }
  } catch {
    if (seq !== requestSeq) return;
    failed = true;
  } finally {
    if (seq === requestSeq) loading = false;
  }
}

console.error(` ✗ ${name}\n ${err.message}`);
process.exitCode = 1;
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 MINOR: Failed assertions leave stale transcript files in the shared corpus.

If a check throws, the per-test fs.rmSync(...) at the bottom of each block does not run. The next test then inherits leftover transcripts and can fail with unrelated usage totals. The existing CodeRabbit thread raised this; it remains unresolved.

Move cleanup into a finally block inside check, and reset the transcript cache there so every test starts from a known state regardless of pass/fail.

Suggested fix:

function check(name, fn) {
  try {
    fn();
    passed++;
    console.log(`  ✓ ${name}`);
  } catch (err) {
    console.error(`  ✗ ${name}\n    ${err.message}`);
    process.exitCode = 1;
  } finally {
    fs.rmSync(CORPUS, { force: true, recursive: true });
    fs.mkdirSync(CORPUS, { recursive: true });
    resetTranscriptCache();
  }
}

Comment thread specs/types/usage.yaml
type: array
description: Per-model rollups, largest token count first
items:
$ref: './specs/types/usage.yaml#/Usage/ModelUsage'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 MINOR: sessions description does not match the implementation.

The description says “most recently active first,” but usage-reader.ts sorts sessions by tokens.totalTokens descending, and UsagePanel.svelte relies on that order for the busiest session. Update the description to “largest token count first” to match the actual contract.

This is the same issue the existing CodeRabbit thread raised; it remains unresolved.

Adds a "Burn" panel to the dashboard answering what the agents are
consuming right now, per session, read from Claude Code transcripts.

Getting the numbers right needed two non-obvious things.

Deduplication. Claude Code splits one API response across several JSONL
entries sharing a message.id, each carrying a usage block. Summing them
all overstated output tokens by 124.6% on a real 11.5k-line session.
Keeping the first entry instead understates, because usage accumulates
as the response streams: 575 of 6681 responses disagreed across their
entries and in every one of those the last entry was larger, worth 4.3%
of output tokens. The last write wins, matching what
docs/CLAUDE-CODE-SESSION-JSONL-SCHEMA.md already documented. requestId
is not a usable key -- it appeared on 1 of 4144 records.

Tail reads. The corpus reached 9.3 GB across 15k files, 1.5 GB of it
touched in a day, mostly subagent transcripts nested under
<session>/subagents/ that a flat readdir misses entirely. Reading files
whole took 21s. Since transcripts are append-only and chronological,
everything inside a trailing window sits at the end of the file, so only
the last 4 MB of files whose mtime falls inside the window are read.
That is why this reports a window rather than all time.

All I/O is async and bounded by a concurrency pool. The synchronous
version blocked the event loop for ~3s on a cold scan, which on a server
whose main job is streaming live terminals would stall every session on
the box. Measured with a 10ms probe running during the scan, the worst
stall is now 22ms, and the cold scan got faster too: 434ms cold, 103ms
warm. Concurrent callers share one in-flight computation instead of each
starting their own walk.

Cost is never guessed. A model with no configured rate reports exact
token counts and renders cost as an em dash, never $0.00, so spend is
not silently under-reported; rates come from SHOOTER_MODEL_PRICING or
~/.shooter/pricing.json, and blank model ids and negative rates are
rejected rather than applied to everything. truncatedFiles surfaces when
the tail budget was short -- including a record larger than the budget,
which parses to nothing and would otherwise read as an empty file.
@murdore
murdore force-pushed the feat/session-usage-monitor branch from 5d07c34 to 6202cf9 Compare August 13, 2026 03:17
* lives at the END of the file: only the last TAIL_BUDGET_BYTES are read,
* and only from files whose mtime falls inside the window.
*
* 3. ASYNC I/O, ALWAYS. This server's main job is streaming live terminals over

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks both -- all six findings were real and are fixed in the amended commit.

CRITICAL: synchronous I/O blocking the event loop. Correct, and the most important one. Every read is now on fs/promises behind a bounded concurrency pool (16), and concurrent callers share one in-flight computation instead of each starting a walk. Measured with a 10 ms interval probe running during a cold scan:

before (sync) after (async)
worst event-loop stall ~3000 ms 22 ms
cold scan 3.2 s 434 ms
warm scan 130 ms 103 ms

Verified that 5 concurrent callers share a single scan.

MAJOR: truncated single-record transcripts not flagged. Real, and the hole was wider than described. A file whose tail contains only the file's trailing newline also slices to an empty string, so it silently read as an empty file rather than a truncated one. Rather than special-case both, the rule is now the precise one: a capped read is known-complete only if it reached back past the cutoff. oldestSeenMs stays Infinity when nothing parsed, so every degenerate case is caught by the same condition. Covered by a test that writes a 5 MB single-record transcript.

Empty model key / negative rates. Both fixed -- a blank id would prefix-match every model and price the whole corpus at one arbitrary rate. Blank ids and negative input/output are rejected; a negative cache rate falls back to the input rate rather than subtracting. Two tests added.

Stale request replacing the current window. Fixed with a request sequence; only the newest response may commit to snapshot, failed or loading.

Session order description. The description was wrong, not the code -- sessions are sorted heaviest-first because the panel answers "what is burning my budget" and the UI sizes its bars against the first row. Description corrected to match.

Test corpus cleanup. Moved into a finally that wipes and recreates the corpus and resets the transcript cache after every check, so a failing assertion can no longer cascade into unrelated failures.

Suite is now 18 checks, all passing; pnpm run check clean across 1015 files; build and live endpoint re-verified.

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Files reviewed: 16 changed files.

New issues raised this run: 1 MAJOR.

Blocking issues: None.

Findings

  • MAJOR — src/routes/api/usage/+server.ts: The route-level response cache is a single global cached/cachedKey pair. Because a request with a different window/limit evicts cached before await-ing the scan, concurrent callers with different keys can overwrite each other’s stored snapshot, and subsequent requests within the 5 s TTL may receive a snapshot for the wrong window. I left an inline comment with a per-key Map fix that preserves the TTL benefit for each window independently.

Already-handled concerns

The existing review threads from CodeRabbit and Tara-ag appear addressed in the amended commit:

  • Synchronous bulk I/O on /api/usage converted to fs/promises with bounded concurrency.
  • Truncated single-record transcripts now flagged via the !complete && oldestSeenMs >= cutoffMs rule.
  • Blank model IDs and negative input/output rates rejected; negative cache rates fall back to input.
  • UsagePanel.svelte request sequencing prevents stale window responses from overwriting the current selection.
  • Test corpus cleanup moved into a finally block.
  • specs/types/usage.yaml session-order description aligned with the heaviest-first implementation.

Security / auth / secrets check

No hardcoded secrets, auth bypasses, command/shell injection paths, path-traversal vectors, or weakened file-permission changes were introduced in this diff. validateAuth runs before any usage computation, and the walker only reads under the configured/projects directory with paths derived from fs.readdir.

Next step

Address the per-key cache isolation in +server.ts, then this is good to merge.

@murdore
murdore merged commit 0ccb151 into release Aug 14, 2026
18 checks passed
@murdore
murdore deleted the feat/session-usage-monitor branch August 14, 2026 19:15
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 1.36.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants