Skip to content

Commit 01cb406

Browse files
authored
feat: session analytics + /stats and /all-stats pages (#65)
## What & why lik-ui had no usage analytics, and mandatory auto-delete (7-day default, daily prune) physically destroys sessions — so the sessions people actually used were exactly the ones we could never learn anything about. This adds durable per-session analytics captured at the one moment the data is both complete and about to be lost: just before deletion. Implements the plan in `docs/plans/2026-07-29-001-feat-session-analytics-stats-page-plan.md` (origin requirements: `docs/brainstorms/2026-07-29-session-analytics-stats-page-requirements.md`). ## How it works - **`session_analytics` table** (new): one durable record per session, keyed by `session_id` with an upsert so capture is exactly-once. No FK — the record intentionally outlives the user/session cascade. `user_email` is denormalized so `/all-stats` can attribute usage "by whom" after the row is gone. - **Capture-before-delete** (`analytics.py`): a single shared step all four deletion paths route through (manual delete, delete-all, prune job, self-heal). Full record when the platform is readable (tokens with cache split, active/wall-clock timing, message/tool/error tallies with per-tool and per-MCP-server breakdown, agent, lifespan, path); flagged local-only record when the read fails or the platform session is already gone. **Never raises — deletion is never blocked by analytics.** - **Lightweight live read**: `beta.sessions.retrieve` gives cumulative usage + timing + status in one call, so the live section is one call per session (no event-stream walk). - **`/stats`** (own sessions, nav-linked) and **`/all-stats`** (all users, unlinked, login-gated only — real access control deferred). Each splits into a live section and a deleted section, with totals and a server-rendered inline-SVG/CSS over-time view. No new frontend dependencies. ## Review findings folded in Correctness + data-integrity review of the diff surfaced and fixed: - **Guarded upsert** so a degraded re-capture (after a 502'd/failed delete retried once the platform session is gone) can't flip a complete record to `incomplete` or corrupt its metrics — a complete capture is authoritative. - **Owner-only self-heal capture** so a non-owner viewer of a shared, platform-gone session doesn't write a phantom "deleted" record for a row that stays live. ## Testing 330 tests pass (+41 new), covering the four-path × read-outcome matrix, exactly-once/idempotency, the guarded-overwrite transitions, both pages' scoping and empty states, and the nav placement. Verified end-to-end render with seeded data. ## ⚠️ Required follow-up: prod DB migration `db/init.sql` uses `CREATE TABLE IF NOT EXISTS`, which will **not** add the new table to the existing `lik-prod-db`. After merge, apply the additive, non-destructive `CREATE TABLE IF NOT EXISTS session_analytics (...)` + its two indexes to prod as a separate step. Happy to help run it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent c9289b6 commit 01cb406

20 files changed

Lines changed: 1881 additions & 10 deletions
Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
---
2+
date: 2026-07-29
3+
topic: session-analytics-stats-page
4+
---
5+
6+
# Session Analytics & `/stats` Page
7+
8+
## Summary
9+
10+
Two analytics pages report per-session usage (sessions over time, tokens, tool use, errors), each split into
11+
live and deleted sessions. Because physical deletion destroys the platform transcript, every session deletion
12+
first writes a durable analytics record — with full usage when it can be read, and flagged with a reason when
13+
it can't — so no deletion ever goes uncounted. Live sessions get their numbers from a lightweight on-demand
14+
platform read.
15+
16+
---
17+
18+
## Problem Frame
19+
20+
lik-ui has no usage analytics today. Token usage is observed only transiently mid-stream and then discarded,
21+
and the sessions table stores almost no usage data. Once a session is physically deleted — which happens
22+
routinely, since auto-delete is mandatory (default 7 days) and runs daily via the prune job — everything about
23+
that session is gone from both the local DB and the Managed Agents platform. That means the sessions people
24+
actually used and let expire, which is most of them, are exactly the ones about which nothing can ever be
25+
learned. An operator wanting to know how much the product is used, by whom, with which agents and integrations,
26+
and at what token cost, currently has no way to find out, and the window to capture it closes permanently at
27+
each deletion.
28+
29+
---
30+
31+
## Actors
32+
33+
- A1. Session owner: a logged-in user viewing analytics for their own sessions on `/stats`.
34+
- A2. Operator: a logged-in user viewing cross-user analytics on `/all-stats` (today any authenticated user;
35+
real access control is deferred).
36+
- A3. Deletion trigger: any of the four code paths that physically delete a session (manual single delete,
37+
delete-all, the prune retention job, and stale-session self-heal). Each must produce an analytics record.
38+
39+
---
40+
41+
## Key Flows
42+
43+
- F1. Capture-before-delete
44+
- **Trigger:** any session deletion (A3), regardless of which of the four paths initiates it.
45+
- **Actors:** A3.
46+
- **Steps:** (1) Before the platform transcript and local row are removed, attempt to read the session's
47+
usage and derived counts from the platform. (2) Write a deleted-session analytics record. (3) If the read
48+
succeeds, the record carries full usage. (4) If the read fails — including the self-heal case where the
49+
platform session is already gone — the record is still written from the session's known local fields and
50+
flagged with the reason capture was incomplete. (5) Proceed with the existing deletion.
51+
- **Outcome:** exactly one analytics record exists per deleted session; deletion is never blocked by
52+
analytics.
53+
- **Covered by:** R5, R6, R7, R8, R9.
54+
55+
- F2. View analytics
56+
- **Trigger:** A1 opens `/stats`, or A2 opens `/all-stats`.
57+
- **Actors:** A1, A2.
58+
- **Steps:** (1) The page shows two sections: live sessions and deleted sessions. (2) Live-session numbers
59+
are read on demand from the platform (lightweight). (3) Deleted-session numbers come from the stored
60+
analytics records. (4) `/stats` is scoped to the viewer's own sessions; `/all-stats` spans all users.
61+
- **Outcome:** the viewer sees usage over time and totals across both live and deleted sessions.
62+
- **Covered by:** R1, R2, R3, R4, R10, R11, R12.
63+
64+
---
65+
66+
## Requirements
67+
68+
**Pages and access**
69+
- R1. Add a `/stats` page showing analytics for the logged-in viewer's own sessions only.
70+
- R2. Add a "Stats" link to the top navigation, positioned immediately after the "Settings" link, pointing to
71+
`/stats`.
72+
- R3. Add an `/all-stats` page showing analytics across all users' sessions. It is not linked in the
73+
navigation and is reached only by typing the URL.
74+
- R4. `/all-stats` requires a logged-in user but applies no further access restriction in this version;
75+
proper access control is explicitly deferred (see Scope Boundaries).
76+
77+
**Analytics capture on deletion**
78+
- R5. Every session deletion writes exactly one durable analytics record before the session is physically
79+
removed, so the data survives deletion of the platform transcript and local row.
80+
- R6. The capture is a single shared step that all four deletion paths route through: manual single delete,
81+
delete-all, the prune retention job, and stale-session self-heal.
82+
- R7. Each analytics record captures, when readable: token usage (input, output, and cache-read /
83+
cache-creation broken out separately), active and wall-clock time, user-message count, AI-message count,
84+
total tool-use count with a per-tool and per-MCP-server breakdown, error count with error types, the agent
85+
used, and the session's lifespan (created and deleted timestamps) and how it was deleted (which path).
86+
- R8. If the pre-delete usage read fails, the record is still written using the session's known local fields
87+
and flagged with the reason capture was incomplete, so missing analytics are counted and visible rather than
88+
silently absent.
89+
- R9. The self-heal path — where the platform session is already gone and nothing can be read — also writes a
90+
flagged record, so even platform-lost sessions are counted.
91+
92+
**What the pages show**
93+
- R10. Each page is split into a live-sessions section (not yet deleted) and a deleted-sessions section.
94+
- R11. Live-session numbers are obtained by a lightweight on-demand read per live session at view time,
95+
yielding cumulative token usage, timing, and status. The heavier per-message / per-tool / per-error tallies
96+
are not shown for live sessions; they appear only once a session has been deleted and captured.
97+
- R12. Each page presents both totals and a time-based view of sessions and tokens (a "sessions per user /
98+
tokens over time" view). On `/stats` the per-user dimension collapses to the single viewer.
99+
100+
---
101+
102+
## Acceptance Examples
103+
104+
- AE1. **Covers R5, R7.** Given a session with recorded turns, when it is deleted through any path with the
105+
platform reachable, then a record exists afterward containing its token usage, timing, message counts,
106+
tool-use breakdown, error detail, agent, and deletion path — and the record is not flagged.
107+
- AE2. **Covers R8.** Given a session that is deleted while the usage read fails, when deletion completes, then
108+
a record still exists, populated from local fields and flagged as incomplete — and the deletion itself
109+
succeeded.
110+
- AE3. **Covers R9.** Given a session dropped by self-heal because the platform already lost it, when the drop
111+
completes, then a flagged "lost before capture" record exists for it.
112+
- AE4. **Covers R11.** Given a viewer with active (undeleted) sessions, when they open `/stats`, then the live
113+
section shows each session's cumulative tokens, timing, and status, and does not show per-tool or
114+
per-message tallies for those live sessions.
115+
- AE5. **Covers R1, R3.** Given two different users each with sessions, when user A opens `/stats`, they see
116+
only their own sessions; when user A opens `/all-stats`, they see sessions from both users.
117+
118+
---
119+
120+
## Success Criteria
121+
122+
- An operator can answer "how much is the product being used, by whom, with which agents, and at what token
123+
cost" from `/all-stats`, including for sessions that have already been deleted.
124+
- No session deletion after this ships is absent from analytics: every deletion yields a record, and any record
125+
that couldn't be fully captured is visibly flagged rather than missing.
126+
- `/stats` loads acceptably for a normal user's own session count without a noticeable per-session read stall.
127+
- A downstream implementer can build from this doc without having to decide the audience/access model, which
128+
metrics to record, where live numbers come from, or which deletion paths capture — all are settled here.
129+
130+
---
131+
132+
## Scope Boundaries
133+
134+
- Real access control / admin roles on `/all-stats` — deferred; this version only requires a logged-in user.
135+
- Persisting live-session usage into the DB or periodic snapshotting — live numbers stay on-demand.
136+
- Full per-event detail (message / tool / error tallies) for live sessions — deleted sessions only.
137+
- Backfill of sessions deleted before this ships — unrecoverable; only post-ship deletions get records.
138+
- Any change to deletion or retention behavior itself (cadence, the mandatory auto-delete, the prune job).
139+
- Export/CSV, dollar-cost pricing, and cross-workspace analytics.
140+
141+
---
142+
143+
## Key Decisions
144+
145+
- Audience split into two pages: `/stats` (own data, navbar-linked) and `/all-stats` (all users, unlinked).
146+
Rationale: the operator wants a cross-user view, but exposing everyone's usage to every user by default is
147+
wrong; keeping the cross-user view unlinked with access control deferred lets the operator use it now without
148+
committing to an admin model that doesn't exist yet.
149+
- Capture happens at deletion time, not via continuous snapshotting. Rationale: the only moment the data is
150+
both complete and about to be lost is right before deletion; a single capture step is far simpler than
151+
keeping a live mirror in sync, and it fits all four existing deletion paths.
152+
- Always write a record, even on capture failure or self-heal. Rationale: a silently missing record looks
153+
identical to "no session existed," which would quietly bias every aggregate; a flagged record keeps the
154+
denominator honest.
155+
- Live sessions use a lightweight read (cumulative usage + timing + status), not the full event tally.
156+
Rationale: the full tally needs a per-event pass per session, which is too costly on every page load; the
157+
cheap read still delivers the tokens-and-time numbers the pages are about.
158+
159+
---
160+
161+
## Dependencies / Assumptions
162+
163+
- The platform exposes, per session, cumulative token usage (including cache-read / cache-creation) and
164+
active/wall-clock timing in a single lightweight read, and exposes the per-event stream needed to tally
165+
message, tool-use (with tool and server identity), and error counts. [Certain — verified against the
166+
installed SDK during the brainstorm.]
167+
- The four deletion paths named in R6 are the complete set today. [Certain — verified in the current
168+
codebase.]
169+
- There is a live production database; schema changes must be applied as non-destructive, additive migrations
170+
(not a drop-and-recreate), and the production DB must be migrated as a separate step from merging the code.
171+
- `/all-stats`' live section may be slow when many sessions exist, since it reads each live session on demand;
172+
acceptable for this version, with caching left as later work.
173+
174+
---
175+
176+
## Outstanding Questions
177+
178+
### Deferred to Planning
179+
180+
- [Affects R7][Technical] Exact shape and storage of the captured record (fields, types, how the per-tool /
181+
per-server breakdown and error types are stored), and the migration to add it.
182+
- [Affects R6][Technical] Where the single shared capture step lives so all four deletion paths — including the
183+
separately-run prune job — invoke it without duplication.
184+
- [Affects R12][Design] The concrete visualization (chart types, table columns, time bucketing) for the
185+
totals and over-time views.
186+
- [Affects R4][User decision, deferred] What the eventual access-control model for `/all-stats` should be
187+
(admin allowlist, role, etc.) when it is time to secure it.

0 commit comments

Comments
 (0)