Skip to content

Latest commit

 

History

History
295 lines (249 loc) · 7.44 KB

README.md

File metadata and controls

295 lines (249 loc) · 7.44 KB

MNIST Deep Learning Exploration 🧠

Python TensorFlow Keras scikit-learn License Maintenance

A comprehensive exploration of deep learning approaches for the MNIST handwritten digit classification task, implementing various neural network architectures using scikit-learn and Keras frameworks.

📖 Table of Contents

🎯 Project Overview

🔬 Feed Forward Network (scikit-learn)

  • Implementation Features
    • Custom dataset preprocessing pipeline
    • Configurable network architecture
    • Flexible train-test split options
    • Comprehensive performance metrics
  • Analysis Capabilities
    • Iteration impact assessment
    • Split ratio optimization
    • Parameter sensitivity analysis
    • Training time evaluation

🚀 Deep Neural Networks (Keras)

  • Architecture Exploration
    • Variable node configurations (4-2056)
    • Flexible layer depths (4-16)
    • Multiple activation functions
    • Custom layer combinations
  • Design Optimization
    • Hyperparameter tuning
    • Architecture comparison
    • Performance benchmarking
    • Resource utilization analysis

🛠 Technical Architecture

System Components

graph TD
    A[MNIST Dataset] --> B[Data Preprocessing]
    B --> C[Feature Engineering]
    C --> D1[scikit-learn Implementation]
    C --> D2[Keras Implementation]
    D1 --> E1[FFN Model]
    D2 --> E2[DNN Model]
    E1 --> F[Performance Analysis]
    E2 --> F
    F --> G[Results Visualization]
Loading

Dependencies

# requirements.txt
numpy>=1.20.0
matplotlib>=3.4.0
scikit-learn>=1.0.0
tensorflow>=2.8.0
keras>=2.8.0
pandas>=1.3.0
seaborn>=0.11.0

💻 Installation & Setup

System Requirements

  • Minimum Specifications
    • Python 3.8+
    • 8GB RAM
    • 4GB GPU memory
    • 10GB storage
  • Recommended Specifications
    • Python 3.9+
    • 16GB RAM
    • 8GB GPU memory
    • CUDA-compatible GPU
    • 20GB SSD storage

Quick Start

# Clone repository
git clone https://github.com/yourusername/mnist-deep-learning-exploration.git

# Navigate to project
cd mnist-deep-learning-exploration

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Linux/Mac
.\venv\Scripts\activate   # Windows

# Install dependencies
pip install -r requirements.txt

Configuration

# config.py
CONFIG = {
    'data': {
        'train_test_splits': [0.6, 0.75, 0.8, 0.9],
        'batch_size': 32,
        'validation_split': 0.1
    },
    'model': {
        'node_counts': [4, 32, 64, 128, 512, 2056],
        'layer_depths': [4, 5, 6, 8, 16],
        'activations': ['relu', 'sigmoid', 'tanh']
    },
    'training': {
        'epochs': 50,
        'early_stopping_patience': 5,
        'learning_rate': 0.001
    }
}

🔬 Implementation Details

Feed Forward Network

def create_ffn_model(input_dim, hidden_layers, nodes_per_layer):
    """
    Creates a Feed Forward Neural Network using scikit-learn.
    
    Args:
        input_dim (int): Input dimension
        hidden_layers (int): Number of hidden layers
        nodes_per_layer (int): Nodes in each hidden layer
        
    Returns:
        MLPClassifier: Configured neural network
    """
    return MLPClassifier(
        hidden_layer_sizes=(nodes_per_layer,) * hidden_layers,
        activation='relu',
        solver='adam',
        max_iter=200,
        random_state=42
    )

Deep Neural Network

def create_dnn_model(input_shape, architecture):
    """
    Creates a Deep Neural Network using Keras.
    
    Args:
        input_shape (tuple): Shape of input data
        architecture (dict): Model architecture configuration
        
    Returns:
        Model: Compiled Keras model
    """
    model = Sequential()
    
    # Input layer
    model.add(Dense(architecture['nodes'][0], 
                   input_shape=input_shape,
                   activation=architecture['activations'][0]))
    
    # Hidden layers
    for nodes, activation in zip(architecture['nodes'][1:],
                               architecture['activations'][1:]):
        model.add(Dense(nodes, activation=activation))
    
    # Output layer
    model.add(Dense(10, activation='softmax'))
    
    model.compile(optimizer='adam',
                 loss='categorical_crossentropy',
                 metrics=['accuracy'])
    
    return model

📊 Experiments & Results

Node Count Analysis

Nodes Accuracy Training Time (s) Parameters
4 85.2% 12.3 3,214
32 92.1% 15.7 25,962
64 94.5% 18.2 51,914
128 96.2% 22.8 103,818
512 97.1% 35.6 415,242
2056 97.3% 89.4 1,661,962

Layer Depth Study

def plot_depth_performance():
    """
    Visualizes performance across different network depths.
    """
    plt.figure(figsize=(10, 6))
    plt.plot(depths, accuracies, marker='o')
    plt.xlabel('Network Depth')
    plt.ylabel('Accuracy')
    plt.title('Performance vs Network Depth')
    plt.grid(True)

⚡ Performance Analysis

Optimization Techniques

  • Batch normalization
  • Dropout layers
  • Learning rate scheduling
  • Early stopping

Benchmarks

Architecture Accuracy Training Time Memory Usage
Basic FFN 92.5% 25s 450MB
4-Layer DNN 95.8% 45s 680MB
8-Layer DNN 97.2% 78s 920MB
16-Layer DNN 97.5% 156s 1.2GB

👨‍💻 Development

Project Structure

mnist-deep-learning/
├── data/
│   ├── raw/
│   └── processed/
├── models/
│   ├── ffn/
│   └── dnn/
├── src/
│   ├── preprocessing.py
│   ├── ffn_model.py
│   ├── dnn_model.py
│   └── visualization.py
├── notebooks/
│   ├── exploration.ipynb
│   └── analysis.ipynb
├── tests/
│   └── test_models.py
├── config.py
├── requirements.txt
└── README.md

Testing

# Run all tests
python -m pytest

# Run specific test file
python -m pytest tests/test_models.py

# Run with coverage
python -m pytest --cov=src

🤝 Contributing

Workflow

  1. Fork repository
  2. Create feature branch
  3. Implement changes
  4. Add tests
  5. Submit pull request

Code Style Guidelines

  • Follow PEP 8
  • Document all functions
  • Write comprehensive tests
  • Maintain clean notebook outputs

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • MNIST Dataset creators
  • TensorFlow and Keras teams
  • scikit-learn community
  • CSET-335 course staff