DSLR - DataScience × Logistic Regression

A 42 School data science project: a multi-class logistic regression classifier that sorts Hogwarts students into one of the four houses - Gryffindor, Slytherin, Ravenclaw, Hufflepuff - based on their course scores.

The machine learning pipeline is implemented from scratch (no ready-made sklearn classifier): sigmoid, cross-entropy loss and gradient descent by hand, using a one-vs-all strategy across the four houses.

---

Contents

• About
• Dataset
• Installation
• Usage
• Data Visualization
• The Model
• Project Structure

---

About

The goal is to solve a classic classification problem across the full data science workflow:

1. Data exploration - descriptive statistics with a custom describe implementation (mean, std, min, max, quartiles - a re-write of pandas.describe()).
2. Visualization - histograms, scatter plot and pair plot to understand which courses separate the houses well, and which ones are redundant.
3. Training - logistic regression via gradient descent, one-vs-all.
4. Prediction - using the trained weights to predict the houses of the test set.

Dataset

Students (train)	1600
Houses	Gryffindor, Slytherin, Ravenclaw, Hufflepuff
Features	13 course scores + Best Hand
Target	Hogwarts House

The courses: Arithmancy, Astronomy, Herbology, Defense Against the Dark Arts, Divination, Muggle Studies, Ancient Runes, History of Magic, Transfiguration, Potions, Care of Magical Creatures, Charms, Flying.

Installation

git clone <repo-url>
cd dslr
pip install -r requirements.txt

Dependencies: numpy, pandas, matplotlib, seaborn, scikit-learn, joblib.

Usage

The project is driven by a Makefile:

make all            # the full pipeline: describe → charts → train → predict
make describe       # descriptive statistics to the terminal
make histogram      # histograms into the results/ folder
make scatter_plot   # scatter plot into the results/ folder
make pair_plot      # pair plot into the results/ folder
make train          # train the logistic regression model
make predict        # prediction → results/houses.csv
make clean          # remove generated files
make help           # list all available targets

The results/ folder is not version-controlled - the charts are created at run time. make bonus runs the extended statistics and the bonus (SGD/mini-batch) training.

Data Visualization

The point of the visualization step is to discover which features carry information about the house assignment. The three charts below are the most telling results.

1. Scatter plot - two perfectly correlated features

There is a perfect negative linear relationship between Astronomy and Defense Against the Dark Arts - the points fall on a single straight line. This is the key insight of the project: the two courses effectively carry the same information, so one of them is redundant and can be dropped from training. Uncovering exactly this kind of relationship is what the visualization is for.

2. Histogram - a feature that separates the houses

The distribution of Astronomy scores splits cleanly by house: two barely-overlapping groups emerge. A course with such a bimodal, well-separated distribution is a strong discriminative feature - it helps the model guess the right house. In contrast, a homogeneous, fully-overlapping course (e.g. Care of Magical Creatures or Arithmancy) carries little information.

3. Pair plot - an overview of the whole feature space

The pair plot charts every numeric feature against every other, colored by house. In a single image you can read off which courses separate the houses well (distinct color clusters), which are redundant (pairs falling on a line, like Astronomy-DADA), and which are noisy. This is what informs the choice of features selected for training.

The Model

The classifier is built from scratch, without an sklearn classifier:

• Sigmoid activation: 1 / (1 + e^-z)
• Cross-entropy (log-loss) cost function
• Gradient descent: learning_rate = 0.01, 1000 iterations
• One-vs-all: four binary classifiers, one per house - the prediction picks the house with the highest probability
• Standardization: StandardScaler (the scaler is saved to models/scaler.pkl for consistent prediction)

The trained weights are written to models/weights.txt; prediction produces results/houses.csv in Index,Hogwarts House format.

Bonus

• describe_bonus - extended statistics (variance, skewness, etc.)
• logreg_train_bonus - SGD / mini-batch gradient descent variant

Project Structure

dslr/
├── data/
│   ├── dataset_train.csv      # training data (1600 students, labeled)
│   └── dataset_test.csv       # test data (for prediction)
├── scripts/
│   ├── describe.py            # descriptive statistics (pandas.describe re-write)
│   ├── describe_bonus.py      # extended statistics
│   ├── histogram.py           # course distributions by house
│   ├── scatter_plot.py        # Astronomy vs DADA
│   ├── pair_plot.py           # scatter plot matrix
│   ├── logreg_train.py        # training (gradient descent)
│   ├── logreg_train_bonus.py  # training (SGD / mini-batch)
│   └── logreg_predict.py      # prediction
├── models/                    # trained weights + scaler (generated)
├── results/                   # charts + houses.csv (generated)
├── docs/                      # README figures
├── Makefile
└── requirements.txt

---

_{42 School · DataScience × Logistic Regression}