Detection of Diffusion Model Generated Images / Videos

Final Video Link: https://drive.google.com/file/d/1f30i8VRXWJACDL4tsARPlDs-sRP1FRl7/view?usp=sharing

Overview

This project provides a comprehensive pipeline for detecting images generated by diffusion models versus real/authentic images. It implements various machine learning techniques including CNNs, feature extraction, and robustness evaluation against image transformations. The project is organized into modular components for easy maintenance, extension, and reproducibility.

Project Features

• Model Training & Evaluation: Train deep learning models to classify diffusion-generated vs. real images
• Feature Extraction: Extract meaningful features from images for classification
• Robustness Testing: Evaluate model performance under various image transformations (blur, noise, compression, resizing, etc.)
• Video Processing: Extended video analysis module for frame-by-frame detection
• Comprehensive Analysis: ROC curves, confusion matrices, JPEG compression analysis, and more
• Modular Architecture: Well-organized codebase for easy reuse and extension

Directory Structure

├── data/                        # Data storage (raw and processed)
│   ├── raw/                     # Original images (real/fake) and dataset_index.csv
│   └── processed/               # Extracted features (.npy) and train/test splits (.csv)
├── results/                     # Output storage
│   │                           # Metrics, JSON results, and generated plots
├── src/                         # Main source code
│   ├── preprocessing/           # Data download and augmentation
│   │   └── download.py         # Dataset download utilities
│   ├── features/                # Feature extraction and indexing
│   │   ├── build_dataset_index.py
│   │   ├── extract_features_train.py
│   │   ├── extract_features_test.py
│   │   └── split_train_test.py
│   ├── models/                  # Model training and inference
│   │   ├── network.py          # Neural network architecture
│   │   ├── train_and_evaluate.py
│   │   ├── train_generalization.py
│   │   ├── train_robust_detector.py
│   │   ├── train_specific.py
│   │   └── predict.py
│   ├── analysis/                # Evaluation and visualization
│   │   ├── table1_results.py
│   │   ├── table2_jpeg.py
│   │   ├── figure4_roc.py
│   │   ├── confusion_matrix.py
│   │   ├── blur.py
│   │   ├── noise.py
│   │   ├── resize.py
│   │   ├── crop.py
│   │   ├── denoise.py
│   │   └── rigorous_evaluation.py
│   ├── utils/                   # Shared utilities
│   │   ├── augmentation.py
│   │   ├── feature_extraction.py
│   │   └── jpeg_utils.py
│   ├── robust_pipeline/         # Robust detection pipeline
│   │   ├── preprocessing.py
│   │   ├── feature_extraction.py
│   │   ├── model.py
│   │   ├── train.py
│   │   └── predict.py
│   └── config.py               # Centralized path management
├── video_extension/             # Video processing module
│   ├── cli.py
│   ├── main.py
│   ├── config.yaml
│   ├── requirements.txt
│   ├── outputs/                # Model weights and outputs
│   └── video_src/              # Video processing code
│       ├── data/               # Dataset utilities
│       ├── evaluation/         # Evaluation metrics
│       ├── models/             # Video model architectures
│       └── preprocessing/      # Frame extraction and processing
├── synthbuster/                 # SynthBuster related resources
├── cli.py                       # Command-line interface
├── cnn.ipynb                    # Jupyter notebook for CNN experiments
├── requirements.txt             # Python dependencies
└── README.md                    # This file

Installation

Prerequisites

• Python 3.8 or higher
• pip package manager

Setup

1. Clone or download this repository:

cd Diffusion_model_detection

2. Install required dependencies:

pip install -r requirements.txt

3. For video processing module (optional):

cd video_extension
pip install -r requirements.txt
cd ..

Usage

Running Scripts

All scripts are configured to run from the project root directory and use centralized path management via src/config.py:

# Training and evaluating the main model
python src/models/train_and_evaluate.py

# Running specific transformation robustness tests
python src/analysis/blur.py
python src/analysis/noise.py
python src/analysis/resize.py

# Generating evaluation reports
python src/analysis/table1_results.py
python src/analysis/figure4_roc.py

Feature Extraction Workflow

# 1. Build dataset index
python src/features/build_dataset_index.py

# 2. Extract features from training set
python src/features/extract_features_train.py

# 3. Extract features from test set
python src/features/extract_features_test.py

# 4. Split data into train/test
python src/features/split_train_test.py

Video Processing

cd video_extension
python main.py --input <video_path>

Key Components

Path Management

• All scripts use src/config.py for centralized path resolution
• Scripts can be executed from the project root without path issues
• Automatically handles data, results, and model directories

Data Organization

• Raw Data: Original images organized in data/raw/ with real/fake categories
• Processed Data: Extracted features stored as NumPy arrays (.npy)
• Results: Metrics, plots, and evaluation results saved to results/

Model Training

• train_and_evaluate.py: Standard training with validation and testing
• train_generalization.py: Testing model generalization across datasets
• train_robust_detector.py: Training models robust to image transformations
• train_specific.py: Training task-specific models

Robustness Evaluation

Comprehensive testing against common image transformations:

• Blur effects
• Gaussian noise
• JPEG compression
• Image resizing
• Cropping
• Denoising operations

Analysis & Visualization

• ROC curves and AUC metrics
• Confusion matrices
• JPEG quality analysis
• Transformation impact assessment
• Statistical evaluation

Configuration

Edit src/config.py to customize:

• Data directory paths
• Model output locations
• Result storage paths
• Feature extraction parameters

Output & Results

Model training and evaluation produce:

• Trained Models: PyTorch .pth weight files
• Metrics: CSV files with accuracy, precision, recall, F1-score
• Visualizations: PNG plots and confusion matrices
• JSON Reports: Detailed results for reproducibility
• Tables: Summary results matching publication formats