AI-READI Dataset of Type 2 Diabetes Explorer

In this repository we provide code for exploring the Flagship Dataset of Type 2 Diabetes from the AI-READI Project (v3.0.0). It serves as an exploratory dashboard for understanding the dataset structure and quality: raw signal visualization for wearable data and CGM, plus the computed features produced by the preprocessing pipeline.

Note: this explorer covers only the modalities we included (wearables, CGM, and ECG). An explorer for images of the retina is not included in this version.

It includes functionality for:

• Cohort explorer — filter participants on demographics, clinical variables and computed wearable features, then export the matching cohort joined to any participant-level table
• Individual participant view — raw multimodal signals, daily and participant-level computed features, and the pipeline's quality-control record
• Exploratory dashboard as a Streamlit app

⚙️ Installation & Running the Dashboard

First, install the required Python packages using the requirements.txt file:

pip install -r requirements.txt

Once the data is placed in data/raw (see Data Setup below), you can run the dashboard using:

cd src
streamlit run dashboard.py

📁 Repository Structure

• src/dashboard.py: Main entry point for the Streamlit interactive dashboard.
• src/loader.py: Raw JSON / TSV / clinical-CSV parsers, and readers for the pipeline's processed outputs.
• src/config.py: Paths, the pipeline's column contract (modality prefixes, window tokens, output file names), and the per-modality QC thresholds.
• src/cohort_explorer.py & src/visualizations.py: Modules for rendering cohort summaries and multimodal data plots.
• src/qc_tracker.py: Reads the pipeline's qc_tracking.csv, wear_gate.csv and anomaly reports for display.

💾 Data Setup

Researchers are expected to request the AI-READI dataset and place the raw files into the data/raw directory.

The data/raw directory should follow the standard AI-READI structure, containing:

• participants.tsv: Master list of participants and demographics.
• clinical_data/: Contains CSV files like measurement.csv, observation.csv, condition_occurrence.csv, and procedure_occurrence.csv.
• wearable_activity_monitor/: Contains Garmin Vivosmart 5 JSON data for sleep, heart rate, stress, SpO2, respiratory rate, and physical activity.
• wearable_blood_glucose/: Contains Dexcom G6 continuous glucose monitoring JSON data.
• cardiac_ecg/: Contains resting ECG data files.

Processed data

The Daily Computed Features, Summary Features and Data Quality views read the output of the preprocessing pipeline from data/processed. Point that directory at the pipeline's output, or set AIREADI_OUTPUT_ROOT to read a parallel dataset (for example a CGM-coupled subset) without moving files:

AIREADI_OUTPUT_ROOT=/path/to/processed streamlit run dashboard.py

The dashboard reads these files if present, and degrades gracefully when they are not:

file	what it holds
{modality}_all_participants_24h.csv	one row per participant-day, local midnight to midnight
{modality}_all_participants_daynight.csv	one row per participant-night, keyed by wake date
{modality}_participant_summary_{24h,daynight}.csv	one row per participant
sleep_all_participants.csv / sleep_participant_summary.csv	sleep is per-night and has no 24h/day-night split
qc_tracking.csv	per participant and modality, every observation dropped during loading
wear_gate.csv	valid heart-rate minutes per participant-day, and the resulting wear_ok flag
{modality}_anomalies_report.txt	manual-review flags (heart rate, respiratory rate, stress)
participant_covariates.csv, ir_label.csv, cgm_study_features.csv	participant-level clinical covariates, the HOMA-IR label, and CGM study features

Modalities are prefixed sleep hr rr spo2 stress activity cgm, feature columns are named {modality}_{metric}_{window}, and participant-summary columns append the statistic (hr_mean_24h → hr_mean_24h_mean, _sd, _median, _iqr, _min, _max, plus _n). The dashboard groups columns by that grammar rather than by a fixed list, so new features appear without code changes.

Two things worth knowing when reading the tables

• The wear gate. Heart rate is the wear signal — the optical sensor only reads on the wrist. A participant-day with under 60 valid HR minutes is excluded from participant-level aggregates for the 24 h and day windows of every watch-based modality, including activity, whose epoch coverage is logged whether or not the watch is worn. CGM is exempt (the Dexcom G6 is worn independently), and night windows and sleep are not gated, since a scored night is itself wear evidence.
• Nothing is silently discarded. Rows failing a coverage floor stay in the day-level CSVs with their values intact and are flagged {modality}_included_{window} = False; only the summaries exclude them. Every summary row reports both denominators (n_days_total / n_days_used).

Acknowledgements

This code was developed with the help of Gemini 3.1 and Opus 4.6. Please note that this is just a fun side project and should not be used for medical decision making.

License

Code in this repository is released under the MIT License. Data files derived from the AI-READI dataset are subject to the AI-READI Data Use Agreement.