Skip to content

feat(analytics): add churn risk scoring provider - #587

Merged
A6dulmalik merged 5 commits into
MindBlockLabs:mainfrom
storm-beyndtech:feature/churn-risk-provider
Jul 23, 2026
Merged

feat(analytics): add churn risk scoring provider#587
A6dulmalik merged 5 commits into
MindBlockLabs:mainfrom
storm-beyndtech:feature/churn-risk-provider

Conversation

@storm-beyndtech

@storm-beyndtech storm-beyndtech commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Closes #521

Adds GetChurnRiskProvider, which flags users whose activity in the most recent
bucket has dropped sharply against their own historical baseline. Accepts
DateRangeDto with optional granularity and returns a typed
AnalyticsMetricResult.

On the data source

Following up on my comment in the issue, since this shaped the implementation.

Churn risk is a per-user metric, but retention_cohorts is cohort-level and
carries no userId, so it can't produce per-user flags. The fallback is raw
analytics_events, which had no indexes at all — while every other module
indexes this shape (progress.entity.ts has ['userId','attemptedAt']).

So "query pre-aggregated tables where one exists" and "index-backed, no full
table scan" couldn't both be satisfied as the repo stood. I took the cheaper of
the two paths: index analytics_events and aggregate in Postgres, rather than
introduce a per-user rollup entity plus an aggregation job, which is a much
larger change than the two files the issue lists.

If you'd rather have the rollup, the scoring logic transfers unchanged
only fetchBucketCounts gets swapped. Happy to do that in a follow-up, or redo
this PR that way if you prefer it up front.

The query aggregates via date_trunc + GROUP BY, so only one row per
(user, bucket) crosses the wire rather than every raw event.

On the scoring

The hard part isn't the drop calculation, it's that a naive frequency ratio
flags the wrong people. A user doing 40 puzzles a day who drops to 8 reads as an
80% collapse; a user doing one a day who stops entirely reads as a smaller move.

So the drop is divided by that user's own standard deviation. A naturally spiky
player needs a much larger drop to flag than a metronomic one. There's a test
covering exactly this: two users land on an identical recent count, and the
steady one scores 100 while the spiky one scores 27.

Decisions worth pushing back on

  • ACTIVITY_EVENT_TYPES excludes one-shot onboarding events. Including them
    would inflate a new user's first buckets and manufacture a "sharp drop" for
    every account the week after signup. It also excludes xp_awarded and
    user_leveled_up, which are consequences of the actions already counted. This
    is an opinion about the taxonomy — easy to disagree with.
  • Baseline starts at a user's first observed activity, not the range start,
    so accounts aren't penalised for buckets that predate signup. From there,
    silent buckets are materialised as zeros; dropping them would average only the
    active days and hide the decline. Both behaviours have tests.
  • Insufficient history returns null, not 0, matching how cohortSize === 0
    already returns null instead of dividing. A 0 would render as "no risk" for a
    user we know nothing about.
  • Buckets are pinned to UTC so boundaries don't move with the DB session
    timezone, and the JS grid mirrors date_trunc exactly, including Monday-start
    weeks. This is the most fragile part of the change and the bit I'd most want a
    second pair of eyes on.
  • A missing start defaults to a bounded 90-day lookback rather than epoch.
    The existing providers default to new Date(0), which against an unindexed
    table is a scan of everything.

Known limitation

A user who went quiet early in the window scores modestly (22, "low") even
though they're plainly gone — this measures the drop, and by then the
transition already happened outside the comparison. I nearly folded a dormancy
term into the score and deliberately didn't: it would have over-flagged exactly
the spiky users the variance normalisation exists to protect.

Instead consecutiveSilentBuckets is reported unscored, and there's a test
asserting the 22 with a comment explaining why. Sustained dormancy probably
wants its own metric
— worth your call.

Other changes

  • AnalyticsMetricResult.data was hardcoded to RetentionDataPoint[], so it's
    now AnalyticsMetricResult<T = RetentionDataPoint>. The default keeps the
    retention provider, its spec, and the users/retention route compiling
    unchanged.
  • Adds GET /analytics/users/churn-risk behind AnalyticsAdminGuard, matching
    the route added in Add /analytics/users/retention endpoint #527. Because the result type is now generic, the response
    schema is declared with ApiExtraModels + getSchemaPath rather than
    @ApiResponse({ type: ... }) — the latter would advertise RetentionDataPoint
    as the element type. Happy to align both routes on one style if you prefer.
  • Drops an unused ApiQuery import while restructuring that import block.

Rebased onto main after #588, #586 and #583. The only conflict was
analytics.module.ts, where #588 added AnalyticsService to the same arrays.

Verification

  • tsc --noEmit — clean
  • jest src/analytics — 30/30 passing, 4 suites (17 new, 13 existing). Both
    existing analytics specs still pass, so the generic change didn't break
    retention.
  • eslint on the changed files — clean

Heads-up on lint

npm run lint is eslint "{src,apps,libs,test}/**/*.ts" --fix--fix is
baked into the script. On Windows with core.autocrlf=true it rewrites line
endings across the entire backend and produces a ~170-file diff. It's a footgun
for any Windows contributor, given CONTRIBUTING tells everyone to run it before
a PR. Might be worth splitting into lint and lint:fix.

Separately, lint exits 1 on main already — 112 pre-existing errors in
health.*, main.ts and others, none in this PR's files. So a red lint check
here isn't from these changes.

@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

@storm-beyndtech is attempting to deploy a commit to the aminubabafatima8-gmailcom's projects Team on Vercel.

A member of the Team first needs to authorize it.

@A6dulmalik

Copy link
Copy Markdown
Member

Kindly resolve conflict and have the CI checks pass too. Thank you
@storm-beyndtech

analytics_events shipped without any index, so every reporting query over
it is a sequential scan. Adds the two composite indexes the analytics
providers actually need:

- (timestamp, userId) — range scan over a reporting window grouped by
  user, which is the churn risk access pattern and the one a future
  per-user rollup job would use.
- (userId, timestamp) — a single user's history in time order.

Migration guards on table existence and uses CREATE INDEX IF NOT EXISTS,
since analytics_events has no migration of its own and is created by
synchronize. Builds are non-concurrent because TypeORM runs migrations
inside a transaction.

Refs MindBlockLabs#521
…sult

AnalyticsMetricResult.data was hardcoded to RetentionDataPoint[], so no
second metric could return the type the issue asks for. Parameterises it
as AnalyticsMetricResult<T = RetentionDataPoint>; the default keeps the
retention provider and its spec compiling unchanged.

ChurnRiskDataPoint carries the inputs to the score, not just the score,
so a reviewer can see why a user was flagged: baseline mean and standard
deviation, how many buckets backed the baseline, the recent count, and
the trailing silent-bucket run.

riskScore and riskBand are nullable by design. A user with too little
history has no baseline to have dropped from, and reporting 0 there would
render as "safe" on the dashboard for someone we know nothing about.

Refs MindBlockLabs#521
Flags users whose activity in the most recent bucket has dropped sharply
against their own historical baseline.

Scoring is variance-relative rather than ratio-relative. A raw frequency
ratio flags the wrong people: a user doing 40 puzzles a day who drops to
8 reads as an 80% collapse, while a user doing one a day who stops
entirely reads as a smaller move. Dividing the drop by that user's own
standard deviation means a naturally spiky player needs a much larger
drop to flag than a metronomic one.

Notes on the data source: there is no per-user pre-aggregated table in
this module — retention_cohorts is cohort-level and carries no userId —
so this reads analytics_events directly and makes the query index-backed
instead. Aggregation happens in Postgres via date_trunc + GROUP BY, so
only one row per (user, bucket) crosses the wire.

Other decisions worth review:
- Baseline starts at a user's first observed activity, not the range
  start, so accounts are not penalised for buckets predating signup.
  From there, silent buckets are materialised as zeros — dropping them
  would average only the active days and hide the decline.
- ACTIVITY_EVENT_TYPES excludes one-shot onboarding events, which would
  otherwise inflate a new user's first buckets and manufacture a drop the
  week after signup, and system-emitted events, which double-count.
- Buckets are pinned to UTC so boundaries do not move with the DB session
  timezone, and the JS grid mirrors date_trunc exactly (Monday weeks).
- A missing start defaults to a bounded 90-day lookback rather than
  epoch, so an unbounded query cannot become a full scan.

No controller route, matching the GetRetentionCurveProvider precedent.

Refs MindBlockLabs#521
17 cases across the empty, single-day and multi-day ranges the issue asks
for, plus the behaviours that are easy to regress silently:

- an identical drop scores 100 for a steady user and 27 for a spiky one,
  which is the point of normalising by the user's own variance
- silent buckets count as zeros (35, not the 100 you get if they are
  dropped from the baseline)
- the baseline starts at first activity, not the range start
- insufficient history returns null rather than 0
- inverted and single-day ranges return empty without touching the DB

Refs MindBlockLabs#521
Wires the provider to an admin-guarded route, matching the pattern
established by GET /analytics/users/retention in MindBlockLabs#527. Churn scores are
per-user behavioural data, so AnalyticsAdminGuard applies for the same
reason it applies to retention.

AnalyticsMetricResult is generic, so the response schema is declared
explicitly with ApiExtraModels + getSchemaPath. Relying on the class
decorator alone would advertise RetentionDataPoint as the element type,
which is wrong for this endpoint.

Also drops the unused ApiQuery import while restructuring that import
block.

Refs MindBlockLabs#521
@storm-beyndtech
storm-beyndtech force-pushed the feature/churn-risk-provider branch from 18ac521 to a9d61df Compare July 22, 2026 16:19
@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
mind-block-app-frontend Skipped Skipped Jul 22, 2026 4:19pm

@storm-beyndtech

Copy link
Copy Markdown
Contributor Author

Sure @A6dulmalik, done, and you're welcome

@A6dulmalik
A6dulmalik merged commit a0fd346 into MindBlockLabs:main Jul 23, 2026
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Churn risk scoring analytics provider

2 participants