Day 3: Validated NEWS2 scoring and designed ML feature pipelines

- Fixed bug in score_vital: LOC scoring was skipped due to early NaN return.
- Validated GCS/LOC scoring against NHS NEWS2 rules (edge cases now correct).
- Finalised missing data strategies:
  • Timestamp-level: LOCF + missingness flags.
  • Patient-level: median imputation + % missingness.
- Designed pipeline for make_timestamp_features.py (LOCF, flags, rolling windows, staleness, ordinal encoding).
- Designed pipeline for make_patient_features.py (aggregates, median imputation, % missingness, escalation summaries).
- Selected LightGBM as primary ML model (handles NaNs, interpretable, best fit for tabular ICU data).
- Documented all findings, rationales, and lessons in notes.md.

Author: SimonYip22

README.md

@@ -1,7 +1,7 @@
 # EWS-Predictive-Dashboard
 Python-based ICU Early Warning System (EWS) predictive dashboard. Uses time-series vitals to detect patient deterioration, benchmarked against NEWS2, with real-time alerts, trend visualisations, and a dual CLI/FastAPI interface. Portfolio-ready, deployed, and clinically-informed.
 
-
+```text
 Raw ICU Vitals (long format, MIMIC-style)
    └─> compute_news2.py
          ├─ Input: raw vitals CSV
@@ -36,4 +36,5 @@ ML Model (LightGBM)
    │   ├─ Train predictive model for deterioration / escalation
    │   ├─ Use timestamp trends + missingness flags
    │   └─ Evaluate performance (AUROC, precision-recall, etc.)
-   └─ Output: predictions, feature importances, evaluation metrics
+   └─ Output: predictions, feature importances, evaluation metrics
+```

notes.md

@@ -6,25 +6,21 @@
 
 ### Pipeline Overview
 
+```text
 Raw CSVs (chartevents.csv, etc.)
-        │
-        ▼
+        ↓
 extract_news2_vitals.py
-        │
-        ▼
+        ↓
 news2_vitals.csv
-        │
-        ▼
+        ↓
 check_co2_retainers.py
-        │
-        ▼
+        ↓
 news2_vitals_with_co2.csv + co2_retainer_details.csv
-        │
-        ▼
+        ↓
 compute_news2.py
-        │
-        ▼
+        ↓
 Final NEWS2 scores per patient
+```
 
 ### Goals
 - Extract relevant vital signs from PhysioNet.org MIMIC-IV Clinical Database Demo synthetic dataset for NEWS2 scoring.  
@@ -217,18 +213,24 @@ Final NEWS2 scores per patient
 
 ### Overview
 **For timestamp-level ML features (news2_features_timestamp.csv)**:
+
+```text
 raw long vitals (from MIMIC/ICU)  
     ↓ compute_news2.py  
 news2_scores.csv               ← "clinical truth" (all vitals + NEWS2 + escalation labels)  
     ↓ make_timestamp_features.py
 news2_features_timestamp.csv   ← "ML ready" (numeric features, missingness flags, encodings)  
+```
 
 **For patient-level summary features (news2_features_patient.csv)**:
+
+```text
 raw long vitals  
     ↓ compute_news2.py  
 news2_scores.csv                ← news2_patient_summary.csv not needed
     ↓ make_patient_features.py  
 news2_features_patient.csv      ← ML ready (patient-level aggregates, imputed medians, missingness %)
+```
 
 **The difference**:
 - Timestamp pipeline → preserves row-by-row dynamics (LOCF, staleness, rolling windows).