feat(stats): add counter-backed usage and trending stats#746
Open
camathieu wants to merge 6 commits into
Open
Conversation
camathieu
force-pushed
the
improved-stats
branch
12 times, most recently
from
May 1, 2026 18:22
23342fe to
d65e46d
Compare
added 5 commits
July 8, 2026 12:08
…ata layer
One usage_stats row per scope — per-user (id,''), per-token (owner,token),
anonymous ('__anonymous__',''), and a '__deleted__' tombstone that user
deletion folds lifetime counters into so totals never regress. ~50 counters
(current/lifetime uploads, files, sizes, feature/TTL/file-size distributions,
downloads, transferred bytes) are driven by a single registry that generates
the increment updates, the sum-on-read server totals, and the tombstone fold;
a reflection test pins registry <-> schema drift. Daily download/upload
rollups (31-day retention, UTC buckets, user-attributed) power activity
windows, charts, and trending. Every counter mutation runs inside the same
transaction as the metadata change under one canonical lock order (upload row
-> files -> rollups -> user/anon -> token usage), making deadlocks impossible
rather than retried; SQLite opens transactions IMMEDIATE with a busy timeout.
Migration 0011 backfills all counters from existing metadata, with generated
dump fixtures for sqlite3/postgres/mysql/mariadb. The tombstone and rollups
ride exports; everything else rebuilds from metadata on import (fresh-DB
contract). Best-effort single-shot recorders account served/received bytes —
partial transfers included — without ever failing or retrying a request.
…oints
Downloads count through a counting response writer after streaming, so a
client that disconnects mid-stream records exactly its egress; multi-range
requests record the requested range bytes, not multipart framing. Uploads
count wire bytes through a counting reader, including partial and failed
transfers. Archives follow the same partial-bytes policy as single files,
including on mid-archive errors. Recording never blocks, fails, or retries
a user request. New endpoints: GET /stats/activity/daily and
/stats/trending/{uploads,files} (admin), plus self-scoped /me/stats
equivalents including /me/stats/trending/uploads; trending uploads sort by
downloads or downloadedBytes (strict 400 on unknown sort values), rank only
currently-available uploads, and populate both metrics in a single query.
Non-owners never see download counts, bytes, or timestamps (Sanitize).
Daily rollups are pruned with the existing cleaning cycle. The Go client
suite gains stats end-to-end coverage.
fakedb now attributes uploads to the demo admin account, spreads each upload's downloads across the 30-day window with a deterministic FNV-seeded day split (sums stay exact), and seeds per-upload and per-scope transferred bytes consistently with the rollups, so a fresh instance shows a realistic dashboard: populated activity chart, trending panels, and per-user figures.
…ding Admin and Home share the same dashboard components, scoped server-wide or to the current user: an Activity panel whose 4-metric selector (downloads, uploads, downloaded data, uploaded data) drives both the Today/7d/30d/ Lifetime tiles and a hand-rolled SVG daily chart; current-vs-lifetime distribution cards with explanatory captions; and a trending panel with window chips and a downloads/downloaded-data metric toggle, reused self-scoped on Home. Upload cards show total size and downloaded data (owner/admin only), lists sort by downloaded data, and the users tab displays per-user transferred data. Dates and number grouping follow the active locale; byte metrics show an em dash rather than a false zero when the daily series is unavailable; a fresh user gets an encouraging zero-state instead of a wall of zeros. All strings are translated across the 12 locales, guarded by a parity test plus an allowlist-based identical-to-English tripwire.
Covers owner-vs-viewer stat visibility, live sidebar download counts on preview, the activity metric selector and chart tooltips, trending window chips and metric toggle, list sorts, localized rendering, and the Home self-scoped trending panel.
Documents the counter ledger, canonical lock order, sum-on-read totals, tombstone fold, retention, and egress accounting in server/ARCHITECTURE.md; the dashboard component contracts in webapp/ARCHITECTURE.md; every new endpoint, field, sort value, and visibility rule in the API reference; and the requirement that code comments be self-contained and production-ready in the contributor guidelines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
What
Adds backend Stats V1 with DB-backed usage counters, download counters, daily rollups, and admin trending endpoints.
Why
The previous stats path relied on aggregate queries and did not support current vs lifetime usage, trending downloads, token usage stats, or multi-instance-safe counter updates.
Changes
user_usage_stats,server_usage_stats,token_usage_stats, anddownload_stats_daily.startedAt.lastDownloadedAt./me/statsand/statsresponse fields.0011-statsmigration dumps.Testing
git diff --checkmake lintmake client serverNO_RACE=1 make testmake testmake docsmake test-backendspassed on rerunmake vulncompleted, with existing Go 1.26.1 stdlib and npm audit findings reported