🏥 Healthcare Recommendation System

AI-Powered Disease Prediction & Medical Recommendation Platform

A full-stack Big Data application that predicts diseases from patient-reported symptoms using a Spark-trained Random Forest model, served through a FastAPI backend on a Hadoop/HDFS cluster — with a beautiful React dashboard.

🔍 Overview

The Healthcare Recommendation System is an end-to-end Big Data project that combines machine learning, distributed computing, and modern web development to assist users in identifying potential diseases based on their symptoms and providing actionable medical recommendations.

The system uses Apache Spark MLlib to train a Random Forest classifier on a symptom-disease dataset. The trained model is stored in HDFS and served at inference time via a FastAPI REST API. A React frontend provides an intuitive symptom checker dashboard with confidence scores, disease descriptions, and medical precautions.

⚠️ Disclaimer: This system is for educational and research purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. Always consult a qualified healthcare professional.

🏗️ System Architecture

┌─────────────────────────────────────────────────────────────┐
│                        USER BROWSER                         │
│                   React + Vite Frontend                     │
│              (Symptom Checker Dashboard)                    │
└──────────────────────────┬──────────────────────────────────┘
                           │ HTTP / REST API
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                   FastAPI Backend (Python)                   │
│          Disease Prediction & Recommendation API            │
│                    Running on Docker                        │
└───────────┬───────────────────────────────┬─────────────────┘
            │ PySpark                       │ CSV files
            ▼                               ▼
┌───────────────────────┐       ┌───────────────────────────┐
│   Apache Spark        │       │    Recommendation Data     │
│   (ML Inference)      │       │  (Descriptions/Precautions)│
│   spark-master        │       └───────────────────────────┘
│   spark-worker        │
└───────────┬───────────┘
            │ HDFS Read (Model & Labels)
            ▼
┌───────────────────────────────────────────────────────────┐
│                  Apache Hadoop HDFS Cluster                │
│              NameNode  ─────────  DataNode                │
│       /healthcare/models/random_forest  (RF Model)        │
│       /healthcare/models/label_mapping  (Parquet)         │
└───────────────────────────────────────────────────────────┘

Additional Services:
  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐
  │ Apache Hive  │  │   Jupyter    │  │   PostgreSQL         │
  │ (Metastore)  │  │   Notebook   │  │   (Hive Metadata)    │
  └──────────────┘  └──────────────┘  └──────────────────────┘

✨ Features

🤖 AI Disease Prediction — Random Forest model trained with Apache Spark MLlib on 130+ symptoms
📊 Confidence Scoring — Each prediction includes a confidence percentage
💊 Medical Recommendations — Automatically retrieves disease descriptions and precautionary measures
🔬 Big Data Infrastructure — Full Hadoop/HDFS cluster with Apache Spark for distributed ML
📓 Jupyter Integration — Interactive notebook environment connected to the Spark cluster
🐝 Apache Hive — SQL-on-Hadoop metastore for structured data exploration
🎨 Modern React UI — Built with React 19, React Router, and Recharts for data visualization
🐳 Fully Dockerized — One-command deployment with Docker Compose

🛠️ Tech Stack

Layer	Technology
Frontend	React 19, Vite, React Router, Recharts, Axios
Backend API	FastAPI, Uvicorn, PySpark
ML Framework	Apache Spark MLlib (Random Forest)
Distributed Storage	Apache Hadoop HDFS 3.2.1
Data Warehouse	Apache Hive 2.3.2
Metastore DB	PostgreSQL 13
Notebook	Jupyter PySpark (Spark 3.3.0)
Containerization	Docker, Docker Compose

📁 Project Structure

healthcare-project/
├── 📄 docker-compose.yml          # Full stack orchestration
├── 📄 hadoop.env.example          # Hadoop/Hive environment template
│
├── 📂 backend/                    # FastAPI Python backend
│   ├── 📄 Dockerfile
│   ├── 📄 main.py                 # App entrypoint & middleware
│   ├── 📄 requirements.txt
│   ├── 📂 routes/
│   │   ├── 📄 predict.py          # POST /api/predict
│   │   └── 📄 recommend.py        # GET  /api/recommend/{disease}
│   ├── 📂 services/
│   │   ├── 📄 model_service.py    # Spark ML model loader & inference
│   │   └── 📄 recommend_service.py # CSV-based recommendation lookup
│   └── 📂 utils/
│       └── 📄 spark_session.py    # Spark session factory
│
├── 📂 frontend/                   # React + Vite frontend
│   ├── 📄 index.html
│   ├── 📄 vite.config.js
│   ├── 📄 .env.example
│   └── 📂 src/
│       ├── 📄 App.jsx
│       ├── 📂 pages/
│       │   ├── 📄 Home.jsx         # Landing page
│       │   └── 📄 Diagnosis.jsx    # Symptom checker dashboard
│       ├── 📂 components/         # Reusable UI components
│       └── 📂 api/                # Axios API client
│
├── 📂 notebooks/                  # Jupyter notebooks for ML training
├── 📂 scripts/                    # Spark job scripts
├── 📂 data/                       # Raw CSV datasets (gitignored)
└── 📂 models/                     # Trained ML artifacts (gitignored)

🚀 Getting Started

Prerequisites

Make sure you have the following installed:

Docker (v20.10+)
Docker Compose (v2.0+)
At least 8 GB RAM available for Docker (Hadoop + Spark are memory-intensive)
Node.js 18+ (for local frontend development only)

Installation

1. Clone the repository

git clone https://github.com/EZZERBOUTIABDELHAK/Healthcare-Recommendation-System.git
cd Healthcare-Recommendation-System

2. Configure environment variables

# Copy and fill in the Hadoop/Hive configuration
cp hadoop.env.example hadoop.env

# Configure the frontend API URL
cp frontend/.env.example frontend/.env

3. Edit hadoop.env with your Hive metastore password:

HIVE_SITE_CONF_javax_jdo_option_ConnectionPassword=your_secure_password

4. Edit frontend/.env with your backend URL:

VITE_API_URL=http://localhost:8001

Configuration

File	Purpose
`hadoop.env`	Hadoop, HDFS, YARN, and Hive cluster configuration
`frontend/.env`	React app environment (API base URL)

Running the Application

Start all services with Docker Compose:

docker compose up -d

This will spin up:

Service	URL	Description
React Frontend	`http://localhost:5173`	Main web application
FastAPI Backend	`http://localhost:8001`	REST API + Swagger docs
Spark Master UI	`http://localhost:8082`	Spark cluster dashboard
Spark Worker UI	`http://localhost:8081`	Worker node metrics
HDFS NameNode UI	`http://localhost:9870`	Hadoop file system browser
Hive Server UI	`http://localhost:10002`	Hive web interface
Jupyter Notebook	`http://localhost:8888`	PySpark notebooks

Jupyter Token: healthcare123 (change this in docker-compose.yml before deploying publicly)

Run the frontend in development mode (optional):

cd frontend
npm install
npm run dev

Check service health:

docker compose ps
docker compose logs backend --tail=50

Stop all services:

docker compose down

📡 API Reference

The API is auto-documented at http://localhost:8001/docs (Swagger UI).

`GET /`

Health check endpoint.

Response:

{ "status": "Healthcare API is running" }

`POST /api/predict`

Predict a disease from a list of symptoms.

Request Body:

{
  "symptoms": ["fever", "headache", "fatigue", "nausea"]
}

Response:

{
  "disease": "Malaria",
  "confidence": 94.5,
  "label": 12.0
}

`GET /api/recommend/{disease}`

Get the description and precautions for a predicted disease.

Example: GET /api/recommend/Malaria

Response:

{
  "disease": "Malaria",
  "description": "An infectious disease caused by Plasmodium parasites...",
  "precautions": [
    "Consult a doctor",
    "Use mosquito repellent",
    "Avoid stagnant water",
    "Take prescribed antimalarial medication"
  ]
}

📊 Dataset

The model is trained on the Disease Symptom Prediction dataset from Kaggle, which includes:

~4,920 records of symptom-disease pairs
42 diseases across various medical categories
130+ unique symptoms
Supporting CSV files for disease descriptions and precautions

The raw data files are not included in this repository (gitignored). Download them from Kaggle and place them in the data/raw/ directory.

Required files in data/raw/:

data/raw/
├── Training.csv              # Main training dataset
├── symptom_Description.csv   # Disease descriptions
└── symptom_precaution.csv    # Medical precautions

🤖 ML Model

The machine learning pipeline is implemented with Apache Spark MLlib:

Component	Details
Algorithm	Random Forest Classifier
Features	Binary-encoded symptom vector (130+ features)
Training	Distributed training on Spark cluster
Storage	Saved to HDFS as Spark MLlib native format
Inference	PySpark loaded model served via FastAPI

Training Workflow (run in Jupyter):

Load Training.csv into Spark DataFrame
One-hot encode symptoms into binary feature vectors
Label encode diseases
Train RandomForestClassifier (100 trees, max depth 10)
Save model + label mapping to HDFS
Save symptoms_list.json to local /models/ directory

🤝 Contributing

Contributions are welcome! Please follow these steps:

Fork the repository
Create a feature branch: git checkout -b feature/your-feature-name
Commit your changes: git commit -m 'feat: add your feature'
Push to the branch: git push origin feature/your-feature-name
Open a Pull Request

Please make sure your code follows the existing style and that any new features are documented.

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.

Made with ❤️ by EZZERBOUTIABDELHAK

A Big Data & AI project — Healthcare Recommendation System

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🏥 Healthcare Recommendation System

AI-Powered Disease Prediction & Medical Recommendation Platform

📋 Table of Contents

🔍 Overview

🏗️ System Architecture

✨ Features

🛠️ Tech Stack

📁 Project Structure

🚀 Getting Started

Prerequisites

Installation

Configuration

Running the Application

📡 API Reference

`GET /`

`POST /api/predict`

`GET /api/recommend/{disease}`

📊 Dataset

🤖 ML Model

🤝 Contributing

📄 License

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

🏥 Healthcare Recommendation System

AI-Powered Disease Prediction & Medical Recommendation Platform

📋 Table of Contents

🔍 Overview

🏗️ System Architecture

✨ Features

🛠️ Tech Stack

📁 Project Structure

🚀 Getting Started

Prerequisites

Installation

Configuration

Running the Application

📡 API Reference

GET /

POST /api/predict

GET /api/recommend/{disease}

📊 Dataset

🤖 ML Model

🤝 Contributing

📄 License

`GET /`

`POST /api/predict`

`GET /api/recommend/{disease}`