Investigate cohort retention right-censoring bias and contextualizing metrics

[schiste]

Summary

Investigate the right-censoring / time-directionality bias in cohort retention analysis, and design metrics that contextualize "death" rates so cohorts aren't misread as more attrited than they are just because less time has elapsed since their last observed activity.

Context

From a Slack discussion (Lodewijk, 2026-08-22):

Ah for retention we will have to do some corrections probably. There's the implicit bias of time directionality... What would happen if someone who is listed at 2025 as last edit July 2025 and tomorrow they make another edit? (this is generally true for these 'death' rates, there must be tricks to correct)... in a cohort retention analysis, it's considered they never left. So you can have some recovery across time, and we can probably contextualize with some other metrics like "average max interval between edits" to see if your cohort is benefiting from later time return to platform.

The current implementation is build_cohort_output in src/compute/labor.rs:30, which feeds the labor_cohorts output (src/compute/labor.rs:169-172). For a given cohort_year, survived_editors at year Y is derived from last_year >= Y (see the reverse-accumulation loop at src/compute/labor.rs:57-76). That means:

• An editor's cohort is only ever credited with "survival" up to their most recent observed activity — there's no concept of "returned after a gap."
• Recent cohorts (e.g. the current or most recent complete year) haven't had time to demonstrate a return, so their late-period survival looks artificially worse than older cohorts, purely as an artifact of how much calendar time has elapsed — not because they actually churn more.
• If someone's last_year is 2025 and they edit again in 2026, the next full recompute will naturally correct their last_year, but any point-in-time snapshot/dashboard published before that recompute (and any trend analysis built across snapshots) will have already recorded them as attrited for the intervening period.

Proposed scope

• Survey how other retention/survival analyses handle this (e.g. right-censoring adjustments in survival analysis, "resurrected user" tracking in other platforms' growth metrics) and pick an approach that fits our batch/snapshot pipeline model rather than a live-stream model.
• Prototype a "recovery" or "resurrection" view alongside the existing strict survival curve — e.g. does an editor's cohort re-enter the survivor count once they return, versus the current permanent-drop-off framing?
• Add a contextualizing metric such as average max interval between edits per cohort (or per editor, aggregated by cohort) to show whether a cohort's apparent die-off is being offset by long-gap returns — this gives dashboard readers a signal for "this cohort's low late-period survival may partly reflect look-back window, not real churn."
• Decide whether this lives in src/compute/labor.rs alongside build_cohort_output, as a new output table, or as a derived/contextual field on the existing labor_cohorts output.
• Consider whether the same bias affects other places last_year/last edit style logic is used for "activity" or "departure" framing (e.g. src/compute/gdp.rs, docs/wiki-lifecycle.md) — flag if this is broader than just labor cohorts.

Open questions

• Do we correct historical/published snapshots retroactively, or only make future computations resurrection-aware (accepting that older published dashboards remain as they were)?
• What's the right window/threshold for "average max interval between edits" — full history, or a rolling window — and does it need per-wiki calibration (edit cadence varies a lot by wiki size/community)?
• Should resurrection be reflected in labor_cohorts directly (redefining what "survived" means), or exposed as a separate/parallel metric so we don't change the meaning of an existing published column?

Out of scope (for now)

• This is explicitly a "think about later" ticket per the Slack discussion — no urgency to fix before shipping the current retention work. Not blocking any current release.

[schiste]

another way would be to dynamically assign people to a cohort IF they don't leave for more than 2 years, otherwise they join a new cohort on their first edit after n years (would be great if n could be a dynamic variable)