feat(analytics): add churn risk scoring provider - #587
Merged
A6dulmalik merged 5 commits intoJul 23, 2026
Conversation
|
@storm-beyndtech is attempting to deploy a commit to the aminubabafatima8-gmailcom's projects Team on Vercel. A member of the Team first needs to authorize it. |
Member
|
Kindly resolve conflict and have the CI checks pass too. Thank you |
analytics_events shipped without any index, so every reporting query over it is a sequential scan. Adds the two composite indexes the analytics providers actually need: - (timestamp, userId) — range scan over a reporting window grouped by user, which is the churn risk access pattern and the one a future per-user rollup job would use. - (userId, timestamp) — a single user's history in time order. Migration guards on table existence and uses CREATE INDEX IF NOT EXISTS, since analytics_events has no migration of its own and is created by synchronize. Builds are non-concurrent because TypeORM runs migrations inside a transaction. Refs MindBlockLabs#521
…sult AnalyticsMetricResult.data was hardcoded to RetentionDataPoint[], so no second metric could return the type the issue asks for. Parameterises it as AnalyticsMetricResult<T = RetentionDataPoint>; the default keeps the retention provider and its spec compiling unchanged. ChurnRiskDataPoint carries the inputs to the score, not just the score, so a reviewer can see why a user was flagged: baseline mean and standard deviation, how many buckets backed the baseline, the recent count, and the trailing silent-bucket run. riskScore and riskBand are nullable by design. A user with too little history has no baseline to have dropped from, and reporting 0 there would render as "safe" on the dashboard for someone we know nothing about. Refs MindBlockLabs#521
Flags users whose activity in the most recent bucket has dropped sharply against their own historical baseline. Scoring is variance-relative rather than ratio-relative. A raw frequency ratio flags the wrong people: a user doing 40 puzzles a day who drops to 8 reads as an 80% collapse, while a user doing one a day who stops entirely reads as a smaller move. Dividing the drop by that user's own standard deviation means a naturally spiky player needs a much larger drop to flag than a metronomic one. Notes on the data source: there is no per-user pre-aggregated table in this module — retention_cohorts is cohort-level and carries no userId — so this reads analytics_events directly and makes the query index-backed instead. Aggregation happens in Postgres via date_trunc + GROUP BY, so only one row per (user, bucket) crosses the wire. Other decisions worth review: - Baseline starts at a user's first observed activity, not the range start, so accounts are not penalised for buckets predating signup. From there, silent buckets are materialised as zeros — dropping them would average only the active days and hide the decline. - ACTIVITY_EVENT_TYPES excludes one-shot onboarding events, which would otherwise inflate a new user's first buckets and manufacture a drop the week after signup, and system-emitted events, which double-count. - Buckets are pinned to UTC so boundaries do not move with the DB session timezone, and the JS grid mirrors date_trunc exactly (Monday weeks). - A missing start defaults to a bounded 90-day lookback rather than epoch, so an unbounded query cannot become a full scan. No controller route, matching the GetRetentionCurveProvider precedent. Refs MindBlockLabs#521
17 cases across the empty, single-day and multi-day ranges the issue asks for, plus the behaviours that are easy to regress silently: - an identical drop scores 100 for a steady user and 27 for a spiky one, which is the point of normalising by the user's own variance - silent buckets count as zeros (35, not the 100 you get if they are dropped from the baseline) - the baseline starts at first activity, not the range start - insufficient history returns null rather than 0 - inverted and single-day ranges return empty without touching the DB Refs MindBlockLabs#521
Wires the provider to an admin-guarded route, matching the pattern established by GET /analytics/users/retention in MindBlockLabs#527. Churn scores are per-user behavioural data, so AnalyticsAdminGuard applies for the same reason it applies to retention. AnalyticsMetricResult is generic, so the response schema is declared explicitly with ApiExtraModels + getSchemaPath. Relying on the class decorator alone would advertise RetentionDataPoint as the element type, which is wrong for this endpoint. Also drops the unused ApiQuery import while restructuring that import block. Refs MindBlockLabs#521
storm-beyndtech
force-pushed
the
feature/churn-risk-provider
branch
from
July 22, 2026 16:19
18ac521 to
a9d61df
Compare
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
Author
|
Sure @A6dulmalik, done, and you're welcome |
A6dulmalik
approved these changes
Jul 23, 2026
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #521
Adds
GetChurnRiskProvider, which flags users whose activity in the most recentbucket has dropped sharply against their own historical baseline. Accepts
DateRangeDtowith optional granularity and returns a typedAnalyticsMetricResult.On the data source
Following up on my comment in the issue, since this shaped the implementation.
Churn risk is a per-user metric, but
retention_cohortsis cohort-level andcarries no
userId, so it can't produce per-user flags. The fallback is rawanalytics_events, which had no indexes at all — while every other moduleindexes this shape (
progress.entity.tshas['userId','attemptedAt']).So "query pre-aggregated tables where one exists" and "index-backed, no full
table scan" couldn't both be satisfied as the repo stood. I took the cheaper of
the two paths: index
analytics_eventsand aggregate in Postgres, rather thanintroduce a per-user rollup entity plus an aggregation job, which is a much
larger change than the two files the issue lists.
If you'd rather have the rollup, the scoring logic transfers unchanged —
only
fetchBucketCountsgets swapped. Happy to do that in a follow-up, or redothis PR that way if you prefer it up front.
The query aggregates via
date_trunc+GROUP BY, so only one row per(user, bucket) crosses the wire rather than every raw event.
On the scoring
The hard part isn't the drop calculation, it's that a naive frequency ratio
flags the wrong people. A user doing 40 puzzles a day who drops to 8 reads as an
80% collapse; a user doing one a day who stops entirely reads as a smaller move.
So the drop is divided by that user's own standard deviation. A naturally spiky
player needs a much larger drop to flag than a metronomic one. There's a test
covering exactly this: two users land on an identical recent count, and the
steady one scores 100 while the spiky one scores 27.
Decisions worth pushing back on
ACTIVITY_EVENT_TYPESexcludes one-shot onboarding events. Including themwould inflate a new user's first buckets and manufacture a "sharp drop" for
every account the week after signup. It also excludes
xp_awardedanduser_leveled_up, which are consequences of the actions already counted. Thisis an opinion about the taxonomy — easy to disagree with.
so accounts aren't penalised for buckets that predate signup. From there,
silent buckets are materialised as zeros; dropping them would average only the
active days and hide the decline. Both behaviours have tests.
null, not0, matching howcohortSize === 0already returns null instead of dividing. A 0 would render as "no risk" for a
user we know nothing about.
timezone, and the JS grid mirrors
date_truncexactly, including Monday-startweeks. This is the most fragile part of the change and the bit I'd most want a
second pair of eyes on.
startdefaults to a bounded 90-day lookback rather than epoch.The existing providers default to
new Date(0), which against an unindexedtable is a scan of everything.
Known limitation
A user who went quiet early in the window scores modestly (22, "low") even
though they're plainly gone — this measures the drop, and by then the
transition already happened outside the comparison. I nearly folded a dormancy
term into the score and deliberately didn't: it would have over-flagged exactly
the spiky users the variance normalisation exists to protect.
Instead
consecutiveSilentBucketsis reported unscored, and there's a testasserting the 22 with a comment explaining why. Sustained dormancy probably
wants its own metric — worth your call.
Other changes
AnalyticsMetricResult.datawas hardcoded toRetentionDataPoint[], so it'snow
AnalyticsMetricResult<T = RetentionDataPoint>. The default keeps theretention provider, its spec, and the
users/retentionroute compilingunchanged.
GET /analytics/users/churn-riskbehindAnalyticsAdminGuard, matchingthe route added in Add /analytics/users/retention endpoint #527. Because the result type is now generic, the response
schema is declared with
ApiExtraModels+getSchemaPathrather than@ApiResponse({ type: ... })— the latter would advertiseRetentionDataPointas the element type. Happy to align both routes on one style if you prefer.
ApiQueryimport while restructuring that import block.Rebased onto
mainafter #588, #586 and #583. The only conflict wasanalytics.module.ts, where #588 addedAnalyticsServiceto the same arrays.Verification
tsc --noEmit— cleanjest src/analytics— 30/30 passing, 4 suites (17 new, 13 existing). Bothexisting analytics specs still pass, so the generic change didn't break
retention.
eslinton the changed files — cleanHeads-up on lint
npm run lintiseslint "{src,apps,libs,test}/**/*.ts" --fix—--fixisbaked into the script. On Windows with
core.autocrlf=trueit rewrites lineendings across the entire backend and produces a ~170-file diff. It's a footgun
for any Windows contributor, given CONTRIBUTING tells everyone to run it before
a PR. Might be worth splitting into
lintandlint:fix.Separately, lint exits 1 on
mainalready — 112 pre-existing errors inhealth.*,main.tsand others, none in this PR's files. So a red lint checkhere isn't from these changes.