MM - Supervised Learning

About

Sign language recognition using the Sign Language MNIST dataset (24 classes, A-Z excluding J and Z which require motion).

We tried 3 different models and compared their performance:

Model	Test Accuracy	Notes
Logistic Regression	81.23%	Fast baseline, linear decision boundaries
SVM (optimized)	90.39%	Best performer, polynomial kernel with grid search
Gradient Boosting	88.02%	Ensemble method, sequential decision trees

Most confused pairs: V/U (47 errors), N/M (40 errors), U/R (40 errors) — these signs look really similar to each other.

For documentation, the file presentation.pdf contains the slides we used for our final presentation. (In french)

How to Reproduce

1. Prerequisites

• Python 3.12+
• Jupyter Notebook

2. Setup

Clone the repository and install dependencies:

pip install -r requirements.txt

3. Get the Dataset

Download the Sign Language MNIST dataset from Kaggle:

• sign_mnist_train.csv
• sign_mnist_test.csv

Place both files in the data/raw/ directory.

4. Run the Notebooks

Execute the notebooks in order.

Approach

Models

Logistic Regression - Our starting point. Simple and fast, gives us a baseline to compare the other models against.

SVM - We used grid search to find the best hyperparameters (kernel, degree, C, gamma). The polynomial kernel ended up working the best, reaching ~91% accuracy.

Gradient Boosting - HistGradientBoostingClassifier, which is sklearn's histogram-based implementation. Faster than standard gradient boosting on our 220k augmented dataset.

Pipeline

flowchart TD
    A[Sign Language MNIST Dataset<br/>27k train / 7k test] --> B[Data Exploration<br/>24 classes, class distribution]

    B --> C[Data Preparation]
    C --> D[Label Remapping<br/>0-24 → 0-23]
    D --> E[Pixel Normalization<br/>÷ 255]

    E --> F[Split: Processed & Augmented]
    F --> G[Processed Dataset<br/>27k samples]
    F --> H[Data Augmentation<br/>rotation, noise, brightness]
    H --> I[Augmented Dataset<br/>220k samples 8x]

    G --> J[HOG Extraction<br/>784 px → 144 features]
    I --> K[HOG Extraction<br/>784 px → 144 features]

    J --> L[Cross-Validation<br/>5-fold on processed data]
    L --> M[Model Selection]

    K --> N[Final Training<br/>on augmented + HOG]
    M --> N

    N --> O[Logistic Regression<br/>81.23%]
    N --> P[SVM + Grid Search<br/>90.39%]
    N --> Q[Gradient Boosting<br/>88.02%]

    O --> R[Model Evaluation<br/>Confusion matrix, per-class metrics]
    P --> R
    Q --> R

    R --> S[Best Model: SVM<br/>Polynomial kernel, degree 5]

    T[New Hand Gesture Image] --> U[Preprocessing<br/>grayscale, resize, contrast, invert]
    U --> V[HOG Extraction]
    V --> S
    S --> W[Predicted Letter]

    style A fill:#e1f5ff
    style W fill:#d4edda
    style S fill:#fff3cd
    style P fill:#c8e6c9

Loading

Project Structure

supervised-learning/
├── README.md
├── requirements.txt
├── data/
│   ├── raw/                  # Original Sign Language MNIST CSVs
│   ├── processed/            # Normalized + augmented datasets
│   └── test_images/          # Our own images for testing inference
├── notebooks/
│   ├── 01_data_exploration.ipynb
│   ├── 02_data_preparation.ipynb
│   ├── 03_model_training.ipynb
│   ├── 04_model_evaluation.ipynb
│   └── 05_inference.ipynb    #TODO: Inference for testing on new images
├── src/
│   ├── constants.py          # Shared constants
│   └── hog.py                # HOG feature extraction
└── results/
    ├── models/               # Saved trained models (.joblib)
    └── *.png                 # Plots and visualizations

Features

• HOG features: reduces 784 raw pixels to 144 features while capturing shape info
• Data augmentation: 8x with rotation, noise, brightness (no flipping — it changes the meaning of ASL signs)
• Grid search: automatic hyperparameter optimization for SVM
• Full evaluation: confusion matrices, per-class metrics, learning curves

License

This project is licensed under the MIT License - see the LICENSE file for details.

Authors

• Rémi LAVERGNE - remi.lavergne@etu.uca.fr
• Jules LASCRET - jules.lascret@etu.uca.fr

References

Dataset

• Sign Language MNIST

Papers

Research papers that is related to our project: (to complex to implement them but they are interesting)

• Optimization of Transfer Learning for Sign Language Recognition Targeting Mobile Platform - Dhruv Rathi
• Sign Language Recognition with Support Vector Machines and Hidden Conditional Random Fields - De Souza, Cesar & Pizzolato, Ednaldo (2013)

Libraries

• scikit-learn - ML algorithms
• pandas - Data manipulation
• NumPy - Numerical computing
• Matplotlib & Seaborn - Plots
• scikit-image - Image processing (HOG, contrast enhancement)