Healthcare ML DataFusion: Multimodal Diabetes Prediction Pipeline

Developed a Medical AI pipeline that achieves over 98.8% accuracy in predicting diabetes onset by fusing visual deep-learning biomarkers with tabular clinical health records.

---

Summary

This repository is dedicated for a Multimodal Machine Learning diagnostic tool. It bridges the gap between vision-based healthcare (Retinal Fundus Imagery) and traditional tabular clinical records (EHR). By using aunified model that leverages both Convolutional Neural Networks (CNNs) and Tree-based Ensemble Methods, this system demonstrates severe clinical accuracy while generating automated scientific reports—ideal for real-world medical deployments.

Key Achievements

• Multimodal Data Fusion: Engineered a pipeline to robustly integrate 1,536-dimensional image vectors (via Deep Learning) with complex tabular clinical data (via XGBoost), yielding superior predictive performance.
• Deep Learning Feature Extraction: Deployed EfficientNet-B3 (optimized and pre-trained on ImageNet) to reliably extract microscopic biomarkers from retinal scans.
• High-Performance ML Modeling: Built and fine-tuned XGBoost and Random Forest models to interpret metabolic health history efficiently without complex manual feature scaling.
• Automated Performance Reporting: Constructed a reporting sub-system that auto-generates comprehensive medical evaluation metrics (AUC, Sensitivity, Specificity) and formats a professional scientific LaTeX artifact.
• GPU Acceleration: Pipeline dynamically utilizes CUDA-enabled workloads for high-speed model training and image processing.

---

System Architecture

Our solution breaks down medical diagnosis into three distinct processing tracks, ultimately colliding in an SVM-based multimodal classifier:

graph TD
    A[Patient Multi-Source Data] --> B[Retinal Fundus Images]
    A --> C[EHR Clinical Tabular Data]
    B --> D[PyTorch: EfficientNet-B3 on GPU]
    C --> E[XGBoost / Random Forest]
    D --> F[Extracted Visual Biomarkers]
    E --> G[Clinical Risk Factors]
    F --> H[Feature Fusion / Concatenation]
    G --> H
    H --> I[Fusion Classifier SVM]
    I --> J{Diagnosis Prediction}

Loading

---

Project Structure

Healthcare-ml-datafusion/
├── main.py              # Global pipeline execution script
├── models.py            # Deep learning and classical ML model definitions
├── preprocessing.py     # Data loading, cleaning, and preprocessing functions
├── report/              # Directory for automated scientific LaTeX artifact generation
├── .python/             # Python environment configuration
├── .gitignore           # Git ignore file
└── README.md            # Project documentation

---

Evaluation & Impact

By applying multimodal correlation, the system leverages the combined strengths of imaging and tabular data. It achieves near-perfect Sensitivity, which is critical for medical safety applications where false negatives carry significant weight.

Model Modality	Accuracy	F1 Score	Sensitivity	AUC (ROC)
Image-Only (EfficientNet)	97.27%	0.973	96.00%	0.996
Clinical-Only (XGBoost)	99.32%	0.993	99.11%	0.999
Multimodal Fusion Pipeline	98.86%	0.989	97.78%	0.999

All ROC curves, Precision-Recall charts, and detailed evaluation matrices are auto-saved to the results/ directory upon execution.

---

Technology Stack

• Core Languages: Python 3.8+
• Deep Learning Engine: PyTorch / Torchvision
• ML & Data Processing: XGBoost, Scikit-Learn, Pandas, NumPy
• Scientific Output: Matplotlib, Seaborn, Automated LaTeX compiling
• Environment: CUDA-enabled GPU (Nvidia)

---

To Run

1. Clone the repository:

   git clone https://https://github.com/Kri311/Healthcare-Multimodal-Datafusion.git
   cd Healthcare-ml-datafusion

2. Install the environment:

   pip install -r requirements.txt

3. Data Initialization:

   • Map APTOS retinal imagery inside data/aptos/train_images/
   • Map clinical demographic data inside data/diabetes_130/

4. Execute the Global Pipeline:

   python main.py

   Note: This will execute the end-to-end data processing, feature extraction, model fusion, and automated report generation.