Skip to content

README.md

Latest commit

 

History

History
213 lines (164 loc) · 6.65 KB

README.md

File metadata and controls

213 lines (164 loc) · 6.65 KB
Raw

AI-Enhanced User Behavior Analysis Pipeline

An end-to-end analytics pipeline built in Python that simulates and analyzes user engagement data for a digital platform — covering 100,000 sessions across devices, features, cities, and time patterns, enhanced with machine learning capabilities for predictive modeling, clustering, and anomaly detection.

What This Project Does

  • Generates a realistic 100K-row dataset using Faker and NumPy
  • Cleans and preprocesses raw data (outlier removal, type casting, feature engineering)
  • Analyses user behavior across 7 dimensions
  • Produces publication-ready visualizations
  • AI-Enhanced Features:
    • Predictive modeling for session duration (Random Forest)
    • User behavior clustering (K-Means)
    • Anomaly detection (Isolation Forest)
    • Feature importance analysis and model explainability
  • Runs the full pipeline with a single command

Key Insights (Sample Run)

Metric Value
Total Sessions 99,000
Premium Users 24.8%
Avg Session Duration 6.2 mins
Most Used Feature home_feed
Dominant Device Mobile (65%)
Peak Engagement Hour 16:00
Busiest Day Thursday

Project Structure

user_behavior_analysis/
│
├── data/
│   ├── raw/                  # Original generated dataset
│   └── processed/            # Cleaned data
│
├── src/
│   ├── generate_data.py      # Dataset generation
│   ├── data_loader.py        # Cleaning & preprocessing pipeline
│   ├── analysis.py           # All analysis functions
│   ├── visualize.py          # Chart generation
│   └── ml_pipeline.py        # ML/AI components (prediction, clustering, anomaly detection)
│
├── outputs/
│   ├── plots/                # Traditional analytics visualizations (7 charts)
│   └── ml_results/           # ML artifacts and reports
│       ├── models/           # Trained ML models (.pkl files)
│       ├── plots/            # ML-specific visualizations (3 charts)
│       └── reports/          # ML reports (JSON/Markdown)
│
├── main.py                   # Single entry point — runs full pipeline
├── requirements.txt          # All dependencies
└── README.md

📊 Screenshots & Visualizations

Traditional Analytics Visualizations

Hourly Engagement Chart Hourly engagement patterns showing peak usage times

Device Breakdown Chart User distribution across different device types

Day of Week Pattern Weekly engagement patterns and session duration

ML Pipeline Visualizations

ML Feature Importance Feature importance analysis from Random Forest model

ML Cluster Visualization User behavior clusters visualized using PCA dimensionality reduction

ML Model Performance Model performance metrics showing predictive accuracy

Complete Visualization Lists

Traditional Analytics (7 charts)

  • Top Active Users, Hourly Engagement, Feature Popularity
  • Device Breakdown, Premium vs Free, City Engagement, Day of Week Pattern

ML Pipeline (3 charts)

  • Feature Importance, Cluster Visualization, Model Performance

Traditional Analytics Charts

# Chart Description
01 Top Active Users Bar chart of top 10 users by session count
02 Hourly Engagement Line chart of sessions and duration by hour
03 Feature Popularity Usage count and avg duration per feature
04 Device Breakdown Pie + bar chart by device type
05 Premium vs Free Side-by-side behavior comparison
06 City Engagement Session volume by city
07 Day of Week Pattern Sessions and duration by weekday

ML Pipeline Visualizations

# Chart Description
01 Feature Importance Bar chart showing top features driving predictions
02 Cluster Visualization PCA-based 2D visualization of user behavior clusters
03 Model Performance Performance metrics (MAE, RMSE, R²) visualization

ML Pipeline Features

Predictive Modeling

  • Algorithm: Random Forest Regressor
  • Target: Session duration prediction
  • Performance: R² ≈ 0.59, MAE ≈ 2.7 mins
  • Evaluation: 3-fold cross-validation

User Clustering

  • Algorithm: K-Means Clustering
  • Output: 4 distinct user behavior segments
  • Validation: Silhouette score analysis

Anomaly Detection

  • Algorithm: Isolation Forest
  • Detection: Identifies unusual user behavior patterns
  • Rate: ~5% anomaly detection rate

Model Explainability

  • Feature Importance: sklearn-based analysis
  • Top Features: Pages per minute, feature usage, premium status
  • Visualization: Feature importance plots

ML Artifacts Generated

  • Trained models (.pkl files) for deployment
  • ML-specific visualizations (feature importance, cluster plots)
  • Comprehensive reports (JSON + Markdown)
  • Feature importance data (CSV)

Setup & Run

1. Clone the repo

git clone https://github.com/arshsolkar5/user-behavior-analysis.git
cd user-behavior-analysis

2. Create and activate virtual environment

python3 -m venv venv
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

4. Run the full pipeline

python3 main.py

That's it. One command generates the data, cleans it, analyses it, saves all charts, and runs the complete ML pipeline with artifacts and reports.

Output Summary

Traditional Analytics

  • 7 visualization plots saved to outputs/plots/
  • Cleaned dataset saved to data/processed/user_behavior_clean.csv
  • Key insights displayed in terminal output

ML Pipeline Outputs

  • 4 trained models saved to outputs/ml_results/models/
  • 3 ML visualizations saved to outputs/ml_results/plots/
  • Comprehensive reports saved to outputs/ml_results/reports/
  • Feature importance data saved as CSV

Total Artifacts Generated

  • 10 visualization plots (7 traditional + 3 ML-specific)
  • 4 trained ML models (Random Forest, K-Means, Isolation Forest, Scaler)
  • 3 reports (Markdown, JSON, CSV data)
  • 1 cleaned dataset

Tech Stack

  • Python 3.14
  • Pandas — data manipulation
  • NumPy — numerical operations
  • Matplotlib & Seaborn — visualizations
  • Faker — realistic data generation
  • Scikit-learn — machine learning models
  • Standard ML algorithms: Random Forest, K-Means, Isolation Forest

Author

Arsh — Engineering Student