Tardis

Predicting SNCF train delay severity from historical per-route monthly statistics.

Overview

Tardis uses open SNCF data to classify each future month of a given route into one of three delay buckets — Low, Medium, or High — rather than trying to predict the exact delay in minutes. The regression formulation turned out to be too noisy, so the problem was reformulated as a three-class classification task, which produced stable and interpretable results.

The project ships a full pipeline: EDA + cleaning notebook, a modeling notebook that trains and saves an XGBoost classifier, and a Streamlit dashboard for interactive exploration and on-demand prediction.

Data

Source: SNCF open data — "Régularité mensuelle TGV" (monthly TGV punctuality statistics).

Property	Value
Raw rows	10,840 monthly route-level records
Columns	26 (after loading; drops to ~20 after cleaning)
Time range	January 2018 – present (rows beyond today are filtered at load time)
Granularity	One row = one month × one origin–destination pair

Key raw columns: Date, Service, Departure station, Arrival station, Average journey time, Number of scheduled trains, Number of cancelled trains, delay counts at departure/arrival (>15 min, >30 min, >60 min), percentage breakdown of delay causes (external, infrastructure, traffic management, rolling stock, station management, passenger handling).

Approach

Cleaning (tardis_eda.ipynb)

• Parses dates; drops rows outside 2018–today and removes 178 exact duplicates.
• Fuzzy-matches station names and service types to fix typos (threshold 75, fuzzywuzzy).
• Drops columns with >20% missing values and three free-text comment columns.
• Clips the Average delay of late trains at arrival column to the 1st–99th percentile.
• Forward/backward fills remaining gaps then falls back to column mean or mode.

Feature engineering (tardis_eda.ipynb)

New columns created from existing data (no leakage):

Feature	Description
Year, Month, Season	Extracted from Date
Traffic_Intensity	Categorical (Low / Medium / High / Very High) by month
Congestion_Index	Numeric encoding of Traffic_Intensity
Is_School_Holiday	Boolean; months 2, 4, 7, 8, 10, 12
Is_Ski_Period	Boolean; months 2, 3
Is_Bank_Holiday_Period	Boolean; month 5
Journey_Length_Category	Short / Medium-Short / Medium / Medium-Long / Long (by average journey time in minutes)
Train_Density	Low / Medium / High / Very High (by scheduled train count)
Journey_Complexity_Index	Average journey time × Number of scheduled trains / 1000
Route	"DEPARTURE → ARRIVAL" string

Target variable

Average arrival delay is bucketed at load time:

• Low (Faible): ≤ 5 min
• Medium (Moyen): 5 < delay ≤ 15 min
• High (Élevé): > 15 min

The "High" class is a significant minority, making class imbalance a key challenge.

Model (tardis_model.ipynb)

Algorithm: XGBoost (XGBClassifier, objective="multi:softmax")

Pipeline: SimpleImputer(median) + StandardScaler for numerics; SimpleImputer(most_frequent) + OneHotEncoder(handle_unknown="ignore") for categoricals; wrapped in a ColumnTransformer and sklearn.pipeline.Pipeline.

Hyperparameters (found by grid search):

n_estimators=300, max_depth=9, learning_rate=0.1,
subsample=0.8, colsample_bytree=1.0

Custom decision threshold: The probability threshold for predicting "High" is lowered to 0.45 (from the default 0.5) to improve recall on significant delays.

Evaluation: 5-fold stratified cross-validation.

Results

Metric	Value
Best macro F1-score (cross-val)	0.54
Overall accuracy	~0.69

The model notebook outputs are not committed; the figures above come from the hyperparameter search logged in the notebook source. Run the notebooks locally to reproduce.

Project layout

Tardis/
├── tardis_eda.ipynb        # Data cleaning, EDA, feature engineering
├── tardis_model.ipynb      # XGBoost training, evaluation, model export
├── tardis_dashboard.py     # Streamlit app entry point
├── requirements.txt
├── figures/                # Pre-generated EDA charts (PNG)
├── scripts/
│   ├── run_project.sh      # End-to-end runner (venv + notebooks + dashboard)
│   └── clean.sh            # Reset generated artefacts
└── utils/
    ├── data_loader.py      # CSV + model loading helpers
    ├── visualizer.py       # Matplotlib/Seaborn chart functions
    ├── prediction.py       # Feature preparation for inference
    ├── prediction_ui.py    # Streamlit prediction tab
    └── analysis_ui.py      # Streamlit analysis tab

Generated at runtime (not committed):

dataset.csv                 # Raw SNCF data (download separately — see Setup)
cleaned_dataset.csv         # Output of tardis_eda.ipynb
models/
├── tardis_model.joblib     # Trained pipeline (preprocessor + XGBoost)
└── label_encoder.joblib    # LabelEncoder for numeric → category mapping

Setup

git clone https://github.com/SobshDev/Tardis.git
cd Tardis
python -m venv tardis_env && source tardis_env/bin/activate
pip install -r requirements.txt

Download the raw dataset from the SNCF open data portal and save it as dataset.csv (semicolon-separated) in the project root.

Note: The EDA notebook filters out any rows with a date later than today's system date. Make sure your system clock is set correctly before running it.

Usage

Option A — Automatic (recommended)

./scripts/run_project.sh

This installs dependencies, executes both notebooks in order, and launches the dashboard.

Option B — Step by step

# 1. Clean data and engineer features
jupyter notebook tardis_eda.ipynb

# 2. Train and save the model
jupyter notebook tardis_model.ipynb

# 3. Launch the dashboard
python -m streamlit run tardis_dashboard.py

The dashboard opens at http://localhost:8501 and has two tabs: delay analysis (time series, station rankings, seasonal breakdown) and route-level delay prediction.

Using the saved model directly

import joblib, pandas as pd

model = joblib.load("models/tardis_model.joblib")
label_encoder = joblib.load("models/label_encoder.joblib")

new_data = pd.read_csv("your_data.csv", sep=";")
predictions = label_encoder.inverse_transform(model.predict(new_data))

Reset

./scripts/clean.sh

Removes cleaned_dataset.csv, executed notebooks, and the models/ directory.

TODOs

• Integrate real weather data per route and date.
• Add infrastructure maintenance and track condition data.
• Benchmark against other algorithms (LightGBM, neural networks).
• Collect more recent data to extend the time range.
• Publish the trained model artefacts so the dashboard works without running the notebooks first.

Authors

• Gabriel Brument — @SobshDev
• Lohan Lecoq
• Gabin Schiro