Token usage reads the whole transcript: sessions over 2 GiB throw ERR_FS_FILE_TOO_LARGE

[Eliottdupuy]

Summary

getSessionTokenUsage reads the entire JSONL transcript with fs.readFile, but readClaudeTokenUsage only ever needs the tail. On sessions larger than 2 GiB the route throws and never recovers; on every other session it wastes hundreds of MB of I/O per call.

Error

RangeError [ERR_FS_FILE_TOO_LARGE]: File size (3365937082) is greater than 2 GiB
    at readFileHandle (node:internal/fs/promises:538:11)
    at async Object.getSessionTokenUsage (dist-server/server/modules/providers/services/provider-token-usage.service.js:259:33)
    at async dist-server/server/modules/providers/provider.routes.js:557:20 {
  code: 'ERR_FS_FILE_TOO_LARGE'
}

2 GiB is a hard cap in Node's fs.readFile (V8 string / buffer limit), so no amount of heap raises it.

Why the full read is unnecessary

readClaudeTokenUsage (same file, line 82) walks the lines backwards and breaks at the first assistant entry carrying usage:

const lines = fileContent.trim().split('\n');
for (let index = lines.length - 1; index >= 0; index -= 1) {
    const entry = JSON.parse(lines[index]);
    const usage = entry.type === 'assistant' ? entry.message?.usage : null;
    if (!usage) continue;
    ...
    break;   // <- only the tail is ever consulted
}

The result depends only on the end of the file, yet the caller materialises all of it as a single JS string.

Reproduce

1. Run a long Claude Code session until its ~/.claude/projects/<project>/<id>.jsonl exceeds 2 GiB (ours reached 3.37 GB).
2. Open that session in the UI.
3. The token-usage request 500s and the stack above is logged. It repeats on every render.

Version: @cloudcli-ai/cloudcli@1.37.2, Node v22.23.2, Linux arm64.

Suggested fix

Read the tail in growing windows instead of the whole file, and stop as soon as an assistant usage is present. A truncated first line is harmless — readClaudeTokenUsage already skips unparseable lines in its catch.

const TAIL_WINDOWS = [4 * 1024 * 1024, 64 * 1024 * 1024, 512 * 1024 * 1024];

async function readTranscriptTail(filePath) {
    const { size } = await fs.stat(filePath);
    for (const window of TAIL_WINDOWS) {
        if (size <= window) break;
        const handle = await fs.open(filePath, 'r');
        try {
            const buffer = Buffer.allocUnsafe(window);
            const { bytesRead } = await handle.read(buffer, 0, window, size - window);
            const chunk = buffer.subarray(0, bytesRead).toString('utf8');
            if (/"type"\s*:\s*"assistant"/.test(chunk)) return chunk;
        } finally {
            await handle.close();
        }
    }
    return readTextFile(filePath);   // small files keep the existing path
}

Measured on real transcripts

Patched locally and checked against six live sessions. The first 4 MB window was sufficient in all six, including the 3.37 GB file:

session	size	window that sufficed	result
cabb8b7d	3.37 GB	4 MB	33 ms (previously always threw)
82014f46	1.34 GB	4 MB	28 ms, 574 907 tokens
602525ca	0.59 GB	4 MB	30 ms, 815 195 tokens
7f740778	0.57 GB	4 MB	675 389 tokens
92819cd9	0.51 GB	4 MB	603 696 tokens
c66c653c	0.41 GB	4 MB	948 642 tokens

Token counts are identical to those produced by the current full-file path on the sessions where it does not throw.

Happy to open a PR if the approach looks right.