Update

Author: SimonYip22

.DS_Store

@@ -10,10 +10,43 @@
 
 This project implements a dual-architecture early warning system comparing gradient-boosted decision trees (LightGBM) against temporal convolutional networks (TCN) for predicting ICU patient deterioration, across three risk horizons (maximum risk atained, average sustained risk, % time spent in high risk). Built on MIMIC-IV Clinical Demo v2.2 dataset (100 patients), the system processes 171 temporal features across 24-hour windows and 40 aggregated patient-level features, to support continuous monitoring and escalation decisions.
 
+```text
+                               Raw EHR Data
+                   (vitals, observations, lab results)
+                                       │
+             ┌─────────────────────────┴─────────────────────────┐
+             │                                                   │
+             ▼                                                   ▼
+
+   Patient-Level Feature Engineering                 Timestamp-Level Feature Engineering
+(make_patient_features.py → news2_features_patient.csv)     (make_timestamp_features.py → news2_features_timestamp.csv)
+   - Median, mean, min, max per vital                       - Missingness flags 
+   - Imputation using patient-specific median               - Last Observation Carried Forward (LOCF)
+   - % Missingness per vital                                - Carried-forward flags
+   - Encode risk labels and summary target stats            - Rolling windows 1/4/24h (mean, min, max, std, slope, AUC)
+      • max_risk                                            - Time since last observation (staleness)                                  
+      • median_risk                                         - Encode risk labels
+      • pct_time_high
+
+             ▼                                                   ▼
+
+     LightGBM Model (Classical ML)                   Temporal Convolutional Network (TCN)
+   - One fixed-length vector per patient             - Full multivariate sequence per patient per timestamp
+   - Fast, interpretable (SHAP)                      - Learns trends, slopes, sub-threshold drift
+   - Strong baseline for tabular data                - Handles irregular sampling & long-range context
+   - Cannot model sequences                          - Requires sequential data
+```
+
 The hybrid approach reveals complementary strengths: LightGBM achieves superior calibration and regression fidelity (68% Brier reduction, +17% AUC, +44% R²) for sustained risk assessment, while TCN demonstrates stronger acute event discrimination (+9.3% AUC, superior sensitivity) for detecting rapid deterioration. 
 
 The complete pipeline includes NHS-validated NEWS2 preprocessing with CO₂ retainer logic, GCS mapping, and supplemental O₂ protocols; extensive evaluation metrics and model-specific interpretability methods for clinical validation (SHAP for LightGBM, absolute gradient×input saliency for TCN); and a deployment-ready dual inference system (batch and per-patient) for end-to-end usability.
 
+| Target           | Best Model | Key Metric(s)             | Notes |
+|-----------------|------------|--------------------------|-------|
+| Maximum Risk     | TCN        | ROC AUC: 0.923           | Strong acute detection, high sensitivity |
+| Median Risk      | LightGBM   | ROC AUC: 0.972, Brier: 0.065 | Superior sustained risk calibration |
+| Percentage Time High | LightGBM | R²: 0.793                | Better regression fidelity for high-risk exposure |
+
 **Key Contributions:**
 - Clinical validity pipeline with robust NEWS2 computation
 - Dual feature engineering (patient-level vs timestamp) for both classical and deep learning models
@@ -76,6 +109,11 @@ NEWS2 scoring bands map directly to clinical monitoring frequency and escalation
 ### Why Machine Learning?
 ICU deterioration is complex and often subtle, involving multivariate temporal patterns that standard threshold-based systems cannot fully capture. ML models allow us to go beyond static scoring by predicting summary outcomes derived from NEWS2 clinical-risk categories.
 
+| Model | Type | Input Features | Modelling Type | Strengths | Weaknesses | Interpretability |
+|-------|------|----------------|------------------|-----------|------------|----------------|
+| LightGBM | Gradient-Boosted Decision Tree | Aggregated patient-level | Static | Fast, interpretable, good calibration | Cannot capture sequential dynamics | SHAP |
+| TCN | Temporal Convolutional Network | Timestamp-level sequential | Temporal | Captures temporal trends, slopes, variability | Requires high-resolution data, slower to train | Saliency (|grad×input|) |
+
 #### LightGBM (classical, non-temporal ML)
 - LightGBM, a gradient-boosted decision tree (GBDT) algorithm, provides a strong baseline for tabular clinical data
 - Captures nonlinear interactions between vital signs  
@@ -89,7 +127,7 @@ ICU deterioration is complex and often subtle, involving multivariate temporal p
 - Robust to irregular sampling  
 - Potentially detects subtle deterioration earlier than threshold-based approaches  
 
-#### Why compare both
+#### Why compare both?
 - LightGBM provides a robust classical-ML baseline for tabular clinical data.
 - TCN evaluates whether temporal modelling yields measurable gains by capturing sequential patterns and slopes in vital signs.
 - This comparison reflects realistic deployment: classical ML may suffice for lower-frequency ward data, whereas temporal models exploit high-resolution ICU monitoring to detect early deterioration.