READINESS_REDESIGN_PHASE1_FLOORS.md

Readiness Redesign — Phase 1 Baseline Floors

Auto-generated by analysis/phase1_floors/floors.py. Re-run any time to refresh against current target_snapshots + naive_baselines.

Methodology

• Train window (used to BUILD naive baselines by the writers): up to 2024-12-31.
• Test window: 2025-01-01 to 2026-05-15.
• Two test slices:
  • test_2025_current — rows with source_epoch = source_2025_current only. Primary decision basis. 2024 gap (source_2024_gap) is a known source anomaly per the source_epochs catalogue.
  • test_all_post_train — every row in the test window regardless of epoch. Reported for context.
• Continuous: MAE / RMSE per baseline_kind; bootstrap CI on MAE with 1000 resamples.
• Classification: precision at recall = 0.5 (primary), lift over base rate (secondary), AUC (supplementary, not a decision criterion). Stratified bootstrap (positives and negatives resampled separately) preserves class counts on sparse targets like event_strict_t1_t3.

Slice: test_2025_current

Continuous targets — MAE/RMSE floors

recovery_stability / rolling_3d

Eligible test rows: 421

baseline_kind	n	MAE	RMSE	MAE 95% bootstrap CI
persistence_yesterday	404	0.0422	0.0578	[0.0386, 0.0463]
rolling_7d_mean	420	0.0261	0.0341	[0.0239, 0.0283]
rolling_30d_mean	420	0.0253	0.0329	[0.0233, 0.0273]
ewma_45d	420	0.0251	0.0324	[0.0231, 0.0271]

Best naive baseline (lowest MAE): ewma_45d at MAE = 0.0251.

passive_efficiency / rolling_3d

Eligible test rows: 333

baseline_kind	n	MAE	RMSE	MAE 95% bootstrap CI
persistence_yesterday	315	5.3820	7.2855	[4.8505, 5.9466]
rolling_7d_mean	331	3.6197	4.5476	[3.3390, 3.8964]
rolling_30d_mean	331	3.3196	4.1883	[3.0543, 3.5879]
ewma_45d	332	3.1911	4.0647	[2.9263, 3.4626]

Best naive baseline (lowest MAE): ewma_45d at MAE = 3.1911.

Classification targets — precision@recall=0.5 floors

acute_risk / event_t1_t3

Eligible test rows: 401, positives: 122, base rate: 0.3042

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	400	0.328	1.08	0.557	[0.296, 0.368]	61
persistence_yesterday (inline)	393	0.785	2.58	0.848	[0.723, 0.845]	95

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.785 (lift 2.58× over base rate).

acute_risk / event_strict_t1_t3

Eligible test rows: 401, positives: 9, base rate: 0.0224

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	400	0.024	1.07	0.563	[0.022, 0.086]	5
persistence_yesterday (inline)	393	0.667	29.70	0.829	[0.023, 1.000]	6

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.667 (lift 29.70× over base rate).

chronic_load / chronic_label

Eligible test rows: 349, positives: 78, base rate: 0.2235

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	349	0.415	1.86	0.574	[0.191, 0.549]	39
persistence_yesterday (inline)	341	0.947	4.24	0.979	[0.900, 0.987]	72

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.947 (lift 4.24× over base rate).

chronic_load / chronic_acute_density

Eligible test rows: 398, positives: 98, base rate: 0.2462

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	397	0.368	1.49	0.676	[0.323, 0.435]	50
persistence_yesterday (inline)	396	0.918	3.73	0.951	[0.867, 0.967]	90

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.918 (lift 3.73× over base rate).

Slice: test_all_post_train

Continuous targets — MAE/RMSE floors

recovery_stability / rolling_3d

Eligible test rows: 421

baseline_kind	n	MAE	RMSE	MAE 95% bootstrap CI
persistence_yesterday	404	0.0422	0.0578	[0.0386, 0.0463]
rolling_7d_mean	420	0.0261	0.0341	[0.0239, 0.0283]
rolling_30d_mean	420	0.0253	0.0329	[0.0233, 0.0273]
ewma_45d	420	0.0251	0.0324	[0.0231, 0.0271]

Best naive baseline (lowest MAE): ewma_45d at MAE = 0.0251.

passive_efficiency / rolling_3d

Eligible test rows: 333

baseline_kind	n	MAE	RMSE	MAE 95% bootstrap CI
persistence_yesterday	315	5.3820	7.2855	[4.8505, 5.9466]
rolling_7d_mean	331	3.6197	4.5476	[3.3390, 3.8964]
rolling_30d_mean	331	3.3196	4.1883	[3.0543, 3.5879]
ewma_45d	332	3.1911	4.0647	[2.9263, 3.4626]

Best naive baseline (lowest MAE): ewma_45d at MAE = 3.1911.

Classification targets — precision@recall=0.5 floors

acute_risk / event_t1_t3

Eligible test rows: 401, positives: 122, base rate: 0.3042

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	400	0.328	1.08	0.557	[0.296, 0.368]	61
persistence_yesterday (inline)	393	0.785	2.58	0.848	[0.723, 0.845]	95

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.785 (lift 2.58× over base rate).

acute_risk / event_strict_t1_t3

Eligible test rows: 401, positives: 9, base rate: 0.0224

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	400	0.024	1.07	0.563	[0.022, 0.086]	5
persistence_yesterday (inline)	393	0.667	29.70	0.829	[0.023, 1.000]	6

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.667 (lift 29.70× over base rate).

chronic_load / chronic_label

Eligible test rows: 349, positives: 78, base rate: 0.2235

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	349	0.415	1.86	0.574	[0.191, 0.549]	39
persistence_yesterday (inline)	341	0.947	4.24	0.979	[0.900, 0.987]	72

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.947 (lift 4.24× over base rate).

chronic_load / chronic_acute_density

Eligible test rows: 398, positives: 98, base rate: 0.2462

Metric: precision at recall = 0.5, with lift over base rate at the same recall.

baseline_kind	n paired	precision@R=0.5	lift	AUC	precision 95% CI	captured pos
event_base_rate	397	0.368	1.49	0.676	[0.323, 0.435]	50
persistence_yesterday (inline)	396	0.918	3.73	0.951	[0.867, 0.967]	90

Best naive baseline (highest precision@recall=0.5): persistence_yesterday (inline) at precision = 0.918 (lift 3.73× over base rate).

Interpretation note: persistence_yesterday for classification is a window-overlap artifact, not a real floor

persistence_yesterday scores extremely well on every classification target. That does NOT reflect predictive signal. The forward-window labels overlap heavily between adjacent dates:

• event_t1_t3 for date t covers t+1..t+3; for date t+1 covers t+2..t+4. 2 of 3 days shared, so adjacent labels are by construction correlated. Persistence is exploiting label-window overlap, not predicting unseen physiology.
• event_strict_t1_t3 same overlap shape.
• chronic_label and chronic_acute_density look 14 days forward; adjacent labels share 13 of 14 days. Persistence is essentially predicting the overlap, which trivially carries.

The honest classification floor is event_base_rate (the 90-day rolling prior probability). Its lift over base rate at recall = 0.5 is the real benchmark a model must beat to add value.

Decision summary

Floors a future model must beat on the test_2025_current slice. For classification, the floor is event_base_rate (per the note above); persistence is reported in the tables for transparency but is not the decision metric.

target	type	floor (best naive)	floor metric	floor CI half-width	model headroom?
recovery_stability / rolling_3d	continuous	ewma_45d	MAE 0.0251	±0.002	low (close to noise floor) — model needs MAE materially below CI lower bound, target SD 0.033
passive_efficiency / rolling_3d	continuous	ewma_45d	MAE 3.1911	±0.268	low (close to noise floor) — model needs MAE materially below CI lower bound, target SD 4.226
acute_risk / event_t1_t3	binary	event_base_rate	precision 0.328, lift 1.08×	±0.036	low — AUC 0.557; 122 positives in 401 rows
acute_risk / event_strict_t1_t3	binary	event_base_rate	precision 0.024, lift 1.07×	±0.032	insufficient evidence — AUC 0.563; 9 positives in 401 rows
chronic_load / chronic_label	binary	event_base_rate	precision 0.415, lift 1.86×	±0.179	low — AUC 0.574; 78 positives in 349 rows
chronic_load / chronic_acute_density	binary	event_base_rate	precision 0.368, lift 1.49×	±0.056	modest — AUC 0.676; 98 positives in 398 rows

Headroom verdicts derived programmatically:

• Continuous: by MAE / target SD ratio (<0.4 potentially significant, <0.7 modest, otherwise low).
• Classifier: by event_base_rate AUC + positive count (positives<20 insufficient evidence; AUC <0.5 label likely mis-tuned; <0.55 very low; <0.60 low; <0.70 modest; ≥0.70 potentially significant).

Open follow-ups (carried + new)

• Retune chronic_acute_density threshold if its positive rate is far from the 15–30% operationally-useful band. Raise event count threshold; bump chronicLoadFormulaVersion; re-backfill via the admin endpoint; re-run floors. The decision summary above will reflect the change automatically.
• event_strict_t1_t3 is too sparse for an independent classifier while positives stay below ~30. Either accept that strict remains a silent diagnostic only, or relax the strict criterion (e.g. ±1.0σ instead of ±1.5σ) so it becomes informative for Phase 1.
• Investigate 2022 strict event spike (5.7% vs 1–2% in other years). Possible illness cluster / lifestyle change / sensor artifact. Narrative review before feeding into trained models.
• Continuous floors are close to noise floor if MAE / target SD exceeds 0.7. Phase 1 should ask whether the extra features (sleep architecture, intraday HR variability) actually shift this number.
• Persistence is not a useful baseline for forward-window classification labels — the window-overlap math dominates. Either drop the persistence column from the report or change classifier labels to disjoint windows (t+1 instead of t+1..t+3) if persistence-style autocorrelation is needed as a real floor.