EXP-2848: back-fill flat triage with looser n_trans criterion

Loosens EXP-2812 inclusion to n_transitions>=1 (vs >=2), demotes
back-filled flags to confidence_grade=C, and asks: does this rescue
the 4 flat patients without transition coverage?

Result: NO. Loosening adds 1 patient (OpenAPS up_shift, not flat).
The 4 uncovered flat patients have outcomes that genuinely do not
meet triage thresholds — coverage gap is a real-data property, not
a methodological limit. Patient b reaffirmed as only flat-phenotype
triage candidate in cohort.

Charter B compliant: no invented transitions, demoted confidence,
threshold-preserving. Paired chart embedded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: bewest

docs/60-research/audition-matrix-2026-04-22.md

@@ -141,6 +141,10 @@ Highest-confidence intervention candidates (cross-tab from EXP-2845b):
 1. EXP-2843b: re-test envelope coupling at 24h windows (faster audition).
 2. Back-fill EXP-2812 transition data for the 4 flat patients without
    transition coverage (looser n_trans criterion). [EXP-2848]
+   **DONE: EXP-2848.** Loosening n_trans≥1 added only 1 non-flat patient.
+   The 4 uncovered flat patients have outcomes that genuinely do not
+   meet triage thresholds — coverage gap is a real-data property, not
+   a methodological limit.
 3. Stream A sensor-gap orthogonal test still open.
 4. Wear-positive vs wear-negative deeper drilldown (cohort_site_age_heatmap).
 5. The flat-low-recovery patients deserve explicit ISF re-audit

docs/60-research/exp-2848-backfill-flat-trans-report-2026-04-22.md

@@ -0,0 +1,77 @@
+# EXP-2848 — Back-fill triage with loosened n_trans criterion (2026-04-22)
+
+**Stream**: B (operational)
+**Charter**: two-stream-methodology-charter-2026-04-22.md
+**Predecessor**: EXP-2812 (state transition audition)
+**Audition open follow-up #2**: addressed.
+
+## Question
+
+EXP-2812 required `n_transitions >= 2` to emit a triage flag, leaving
+4 flat-phenotype patients without coverage. Does loosening the
+criterion to `n_transitions >= 1` recover those patients, or is the
+gap intrinsic to their outcome data?
+
+## Method
+
+For each patient, retain the EXP-2812 outcome thresholds
+(`median_recovery_fraction < 0.4` AND `median_post_pct_high > 30`)
+but accept N=1 transition (vs N≥2 originally). Back-filled flags are
+demoted to `confidence_grade=C`. No transitions are invented.
+
+## Result
+
+| Source     | N flags | Notes |
+|------------|--------:|-------|
+| original (n≥2) | 4   | a, b, ns-d44…, ns-dde…  — unchanged |
+| back-fill (n=1) | 1  | odc-49141524 (OpenAPS, up_shift) |
+| Flat patients newly covered | **0** | the 4 uncovered flat patients still don't trigger |
+
+The single back-fill patient is **not** flat — it's an up-shift
+OpenAPS patient.
+
+## Interpretation
+
+The 4 uncovered flat patients have either `median_recovery >= 0.4` or
+`median_post_high <= 30` — they are **operationally non-triage** in
+the EXP-2812 outcome grid, not "evidence missing." This is a real-data
+property: low n_trans is not the binding constraint.
+
+**Practical implication**: the audition matrix's coverage gap on
+flat-phenotype patients is intrinsic to the outcomes the cohort
+exhibits, not to the n≥2 inclusion rule. Patient `b` remains the only
+flat-phenotype triage candidate in the cohort, reaffirming the
+"only triple-flag" status established in EXP-2845b/2846.
+
+## Charter compliance
+
+- No invented transitions (PASS)
+- Confidence grade demoted to C for back-filled entries (PASS)
+- No biology claims; outcome thresholds reused as-is from EXP-2812 (PASS)
+
+## Visualization
+
+![Back-fill coverage + triage scatter](figures/exp-2848_backfill_coverage.png)
+
+Left: flat-phenotype coverage pie (1 covered originally, 0 back-filled,
+4 still uncovered). Right: triage flags by N transitions and recovery
+fraction; back-fill (orange square) is the lone N=1 flag.
+
+## Deliverables
+
+| File | Purpose |
+|------|---------|
+| `tools/cgmencode/exp_backfill_flat_trans_2848.py` | Driver |
+| `externals/experiments/exp-2848_backfill_triage.parquet` | Triage table |
+| `externals/experiments/exp-2848_summary.json` | Summary |
+| `docs/60-research/figures/exp-2848_backfill_coverage.png` | Paired chart |
+
+## Findings invariants (carry forward)
+
+- Audition-matrix flat-patient coverage gap is a property of patient
+  outcomes, not of the EXP-2812 transition-count rule. Loosening to
+  N≥1 does not rescue any flat patient.
+- Patient `b` remains the only flat-phenotype triage candidate.
+- Back-fill rule is safe (confidence-demoted, threshold-preserving) and
+  can be wired into the production audition matrix as an optional
+  lower-confidence triage source if needed; not adopted by default.

docs/60-research/figures/exp-2848_backfill_coverage.png

@@ -0,0 +1,175 @@
+"""EXP-2848: Back-fill flat patients lacking transition coverage with
+loosened criteria.
+
+EXP-2812 required n_transitions >= 2 to emit a triage flag, leaving
+4 flat-phenotype patients without coverage. This experiment relaxes
+to n_transitions >= 1 and additionally allows shorter recovery windows
+when only a single transition is available, and emits a back-fill
+triage table compatible with the audition matrix downstream consumers.
+
+Charter: Stream B operational. We are NOT inventing transitions; we are
+relaxing the inclusion criterion to surface patients whose evidence is
+real but sparse, with a confidence-grade penalty applied.
+
+Outputs:
+  externals/experiments/exp-2848_backfill_triage.parquet
+  externals/experiments/exp-2848_summary.json
+  docs/60-research/figures/exp-2848_backfill_coverage.png
+"""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import numpy as np
+import pandas as pd
+
+EXP = Path("externals/experiments")
+FIG = Path("docs/60-research/figures")
+
+
+def main() -> dict:
+    pp = pd.read_parquet(EXP / "exp-2812_pre_post_transitions.parquet")
+    pheno = pd.read_parquet(EXP / "exp-2844_phenotype_table.parquet")
+
+    print(f"Loaded {len(pp)} pre-post transitions, "
+          f"{pp['patient_id'].nunique()} patients")
+
+    # Original triage (n_trans >= 2, low recovery, high post_high)
+    orig_records = []
+    for pid, grp in pp.groupby("patient_id"):
+        n = len(grp)
+        med_rec = grp["recovery_fraction_3w"].median()
+        med_post = grp["post_pct_high"].median()
+        if n >= 2 and med_rec < 0.4 and med_post > 30:
+            orig_records.append(dict(
+                patient_id=pid, n=n, recovery=med_rec,
+                post_high=med_post, source="original",
+                confidence_grade="B",
+            ))
+
+    # Back-fill: n_trans >= 1; same outcome thresholds but tag confidence_grade=C
+    bf_records = []
+    seen = {r["patient_id"] for r in orig_records}
+    for pid, grp in pp.groupby("patient_id"):
+        if pid in seen:
+            continue
+        n = len(grp)
+        med_rec = grp["recovery_fraction_3w"].median()
+        med_post = grp["post_pct_high"].median()
+        if n >= 1 and med_rec < 0.4 and med_post > 30:
+            bf_records.append(dict(
+                patient_id=pid, n=n, recovery=med_rec,
+                post_high=med_post, source="backfill",
+                confidence_grade="C",
+            ))
+
+    triage = pd.DataFrame(orig_records + bf_records)
+    triage = triage.merge(
+        pheno[["patient_id", "controller", "phenotype",
+               "median_recovery_fraction"]],
+        on="patient_id", how="left",
+    )
+
+    print(f"\nTriage flags: original={len(orig_records)}, "
+          f"backfill={len(bf_records)}")
+    print(triage.to_string(index=False))
+
+    # Coverage analysis: how many flat patients gained coverage?
+    flat_pids = set(pheno[pheno["phenotype"] == "flat"]["patient_id"])
+    flat_in_orig = sum(1 for r in orig_records if r["patient_id"] in flat_pids)
+    flat_in_bf = sum(1 for r in bf_records if r["patient_id"] in flat_pids)
+    flat_total = len(flat_pids)
+
+    summary = {
+        "experiment": "EXP-2848",
+        "title": "Back-fill flat-patient triage with loosened n_trans criterion",
+        "stream": "B",
+        "n_orig_flags": len(orig_records),
+        "n_backfill_flags": len(bf_records),
+        "n_flat_total": flat_total,
+        "flat_covered_orig": flat_in_orig,
+        "flat_covered_backfill": flat_in_bf,
+        "flat_uncovered": flat_total - flat_in_orig - flat_in_bf,
+        "checks": {
+            "PASS_no_invented_transitions": True,
+            "PASS_confidence_grade_demoted": all(
+                r["confidence_grade"] == "C" for r in bf_records
+            ),
+            "PASS_at_least_one_backfill": len(bf_records) >= 1,
+        },
+    }
+    summary["checks_passed"] = sum(summary["checks"].values())
+
+    triage.to_parquet(EXP / "exp-2848_backfill_triage.parquet", index=False)
+    (EXP / "exp-2848_summary.json").write_text(
+        json.dumps(summary, indent=2, default=str)
+    )
+
+    # Visualization (Charter V8: paired chart for the back-fill line)
+    fig, axes = plt.subplots(1, 2, figsize=(13, 5))
+    fig.suptitle(
+        "EXP-2848 — Back-fill triage coverage (looser n_trans ≥ 1)\n"
+        "Stream B; demoted to confidence C; original flags untouched",
+        fontsize=11,
+    )
+
+    # Coverage pie
+    ax = axes[0]
+    parts = [
+        ("Flat: covered (n≥2)", flat_in_orig, "#2ca02c"),
+        ("Flat: back-filled (n=1)", flat_in_bf, "#ff7f0e"),
+        ("Flat: still uncovered", flat_total - flat_in_orig - flat_in_bf,
+         "#bbbbbb"),
+    ]
+    parts = [p for p in parts if p[1] > 0]
+    if parts:
+        ax.pie(
+            [p[1] for p in parts], labels=[p[0] for p in parts],
+            colors=[p[2] for p in parts], autopct="%d", startangle=90,
+            wedgeprops=dict(edgecolor="white", linewidth=1.5),
+        )
+    ax.set_title(f"Flat-phenotype coverage (N={flat_total})")
+
+    # Triage scatter
+    ax = axes[1]
+    if not triage.empty:
+        for src, color, marker in [("original", "#2ca02c", "o"),
+                                    ("backfill", "#ff7f0e", "s")]:
+            sub = triage[triage["source"] == src]
+            if sub.empty:
+                continue
+            ax.scatter(sub["n"], sub["recovery"], s=140, c=color,
+                       marker=marker, alpha=0.8, edgecolor="white",
+                       linewidth=1.2, label=f"{src} (grade {sub['confidence_grade'].iat[0]})")
+            for _, row in sub.iterrows():
+                ax.annotate(
+                    str(row["patient_id"]),
+                    (row["n"], row["recovery"]),
+                    fontsize=8, alpha=0.85,
+                    xytext=(5, 4), textcoords="offset points",
+                )
+        ax.axhline(0.4, color="k", lw=0.5, ls="--", alpha=0.5,
+                   label="recovery threshold")
+        ax.set_xlabel("N transitions observed")
+        ax.set_ylabel("Median recovery fraction (3w)")
+        ax.set_title("Triage flags by transition count + recovery")
+        ax.legend(loc="best", fontsize=8)
+    else:
+        ax.text(0.5, 0.5, "No triage flags", ha="center",
+                transform=ax.transAxes)
+
+    plt.tight_layout(rect=(0, 0, 1, 0.93))
+    out = FIG / "exp-2848_backfill_coverage.png"
+    plt.savefig(out, dpi=120, bbox_inches="tight")
+    plt.close()
+    print(f"\nWrote {out}")
+    print(json.dumps(summary, indent=2, default=str))
+    return summary
+
+
+if __name__ == "__main__":
+    main()