Kaggle Playground Series Competition — Binary classification challenge: given a driver's telemetry and race state on any given lap, predict whether they will pit on the very next lap.
- Problem Statement
- Dataset
- Exploratory Data Analysis
- Feature Engineering
- Models
- Ensemble Strategy
- Results
- Project Structure
Formula 1 pit stop timing is one of the most strategically critical decisions in motorsport. Pitting too early sacrifices track position; too late means racing on worn tyres and losing lap time.
┌─────────────────────────────────────────────────────────────────┐
│ For every lap of every driver in every race: │
│ │
│ Input → Telemetry snapshot (tyres, pace, position, etc.) │
│ Output → PitNextLap ∈ {0, 1} │
│ │
│ 0 = Driver continues on current tyres next lap │
│ 1 = Driver will pit at the end of this lap │
└─────────────────────────────────────────────────────────────────┘
Key challenge: The dataset is heavily imbalanced — pit stops are rare events (~10–15% of laps), so a naive classifier can get high accuracy by always predicting 0. The evaluation metric is ROC-AUC, which correctly rewards discrimination ability.
| Column | Type | Description |
|---|---|---|
id |
int | Unique row identifier |
Driver |
str | Driver code (anonymised, e.g. D109) |
Compound |
str | Tyre compound: SOFT / MEDIUM / HARD / INTER / WET |
Race |
str | Grand Prix name |
Year |
int | Season year |
PitStop |
int | Whether a pit stop occurred this lap |
LapNumber |
int | Lap number within the race |
Stint |
int | Current tyre stint number |
TyreLife |
float | Laps driven on current set of tyres |
Position |
int | Current race position |
LapTime (s) |
float | Lap time in seconds |
LapTime_Delta |
float | Lap time change vs previous lap |
Cumulative_Degradation |
float | Cumulative pace loss since stint start |
RaceProgress |
float | Fraction of total race laps completed (0–1) |
Position_Change |
float | Position gained / lost since race start |
PitNextLap |
int | Target variable |
id Driver Compound Race Year PitStop LapNumber Stint TyreLife Position LapTime(s) LapTime_Delta Cumulative_Deg RaceProgress Pos_Change PitNextLap
0 D109 HARD Canadian GP 2022 0 50 2 39.0 8 78.49 -7.56 21.02 0.714 5.0 1
1 D086 HARD Dutch GP 2025 1 27 2 7.0 4 75.10 -32.62 -223.21 0.346 -3.0 0
2 ZON HARD Austrian GP 2022 0 59 3 22.0 13 70.95 -7.54 -100.53 0.819 3.0 1
Class 0 (No Pit) ████████████████████████████████████████████ ~87%
Class 1 (Pit) ██████ ~13%
The ~7:1 imbalance means all models use class-weight balancing (scale_pos_weight = neg/pos).
Compound | p25 p50 p75 | Interpretation
──────────────|──────────────────────|────────────────────────────────
SOFT | 8 12 17 | Pitted well before lap 20
MEDIUM | 12 17 23 | Typical window 12–23 laps
HARD | 17 22 30 | Longest stints, wide window
INTERMEDIATE | 10 14 19 | Weather-driven, shorter window
WET | 7 10 14 | Shortest, usually condition-driven
High Importance ───────────────────────────────────────────── Low Importance
TyreLife ██████████████████████████████ Most predictive — age of rubber
RaceProgress ████████████████████ Strategic timing in race
TyreLife_vs_p50 ██████████████ How far past the median pit lap
Cumulative_Degradation ████████████ Compounding pace loss
InPitWindow ████████ Whether in the typical pit window
LapTime_Delta ██████ Sudden lap time change
PitWindowUrgency ██████ How far past the p75 window
Stint ████ Which stint of the race
Three waves of feature engineering were applied, each building on the previous.
The raw data already contains several engineered signals:
LapTime_Delta— lap-over-lap pace changeCumulative_Degradation— total pace loss across the stintRaceProgress— how far through the race (normalised 0–1)Position_Change— net position gain/loss
For each (Race, Compound) combination, compute from training pit stops:
PitWindow_p25 / p50 / p75 ← TyreLife percentiles when pit stops occurred
Derived features:
TyreLife_vs_p25 = TyreLife - p25 (negative → haven't reached early window)
TyreLife_vs_p50 = TyreLife - p50 (distance from the median)
TyreLife_vs_p75 = TyreLife - p75 (positive → overdue)
InPitWindow = 1 if TyreLife ∈ [p25, p75]
DistFromPitWindow_Center= TyreLife - p50
TyreLife_PctOfWindow = TyreLife / p50 (clipped at 3×)
PitWindowUrgency = max(0, TyreLife - p75) / p75
LapsToWindowCenter = p50 - TyreLife (negative if overdue)
Leakage prevention: pit window statistics are derived only from training rows where
PitStop == 1andTyreLife >= 2(filters out restart stints). Test rows are mapped via the same lookup with compound-level and global fallbacks.
# Compound typical stint lengths (domain knowledge)
cpd_typical = {"SOFT": 12, "MEDIUM": 17, "HARD": 22, "INTERMEDIATE": 14, "WET": 10}
# Key derived features:
TyreLife_pct_typical # how far through the "expected" stint life
PastPitWindow # binary: 1 if TyreLife > typical stint length
TyreLife_overrun # how many laps past the expected window
Compound_hardness # ordinal: SOFT=1, MEDIUM=2, HARD=3Six model families were trained with 5-fold stratified cross-validation, each outputting out-of-fold (OOF) probabilities for ensemble use.
| Model | Architecture | OOF ROC-AUC | Notes |
|---|---|---|---|
| CatBoost | Oblivious trees + ordered boosting | 0.94644 | Best single model |
| HistGBM | Histogram-based gradient boosting | 0.94510 | sklearn's HistGradientBoosting |
| LightGBM | Leaf-wise boosting | 0.9420 | Fast, strong baseline |
| XGBoost | Level-wise boosting | 0.8750 | Slower convergence |
| BiLSTM | Bidirectional LSTM | 0.8100 | Sequence model on lap data |
| Logistic Regression | Linear | ~0.87 | Diversity/calibration anchor |
Training data
│
├─ Fold 1 ──▶ Train on F2+F3+F4+F5, validate on F1
├─ Fold 2 ──▶ Train on F1+F3+F4+F5, validate on F2
├─ Fold 3 ──▶ Train on F1+F2+F4+F5, validate on F3
├─ Fold 4 ──▶ Train on F1+F2+F3+F5, validate on F4
└─ Fold 5 ──▶ Train on F1+F2+F3+F4, validate on F5
Each fold → OOF probabilities saved as .npy for ensemble
Final test predictions = average of 5 fold models
Iterations : 2000 (early stopping @ 100)
Depth : 6 (symmetric/oblivious trees)
Learning rate : 0.10
L2 leaf reg : 3.0
Scale pos wt : neg / pos (handles class imbalance)
Eval metric : AUC
CatBoost 5-Fold CV Results:
Fold 1 AUC = 0.94712
Fold 2 AUC = 0.94589
Fold 3 AUC = 0.94601
Fold 4 AUC = 0.94700
Fold 5 AUC = 0.94618
─────────────────────────────
OOF AUC = 0.94644 ✓
The following evaluation plots are generated for each model:
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ ROC Curve │ │ F1 vs Threshold │ │ Confusion Matrix │
│ │ │ │ │ │
│ ╭─── │ │ ╭──╮ │ │ TN │ FP │
│ ╱ │ │ ╭╯ ╰╮ │ │──────┼────── │
│ ╱ AUC=0.946 │ │ ─╯ ╰── │ │ FN │ TP │
│ ╱ │ │ 0.0 0.5 1.0 │ │ │
└──────────────────┘ └──────────────────┘ └──────────────────┘
Saved as: catboost_evaluation.png, lgbm_evaluation.png, etc.
Three ensemble versions were built, progressively more sophisticated.
blend = Σ (AUC_i / Σ AUC_j) × predictions_i
Step 1: Convert each model's raw probabilities to uniform ranks
rank_avg(p) = rank(p) / (N + 1) ← removes calibration differences
Step 2: Weight by AUC^5 (power = 5 strongly amplifies top models)
w_i = AUC_i^5 / Σ AUC_j^5
Step 3: Weighted sum of rank-averaged predictions
Model weights (v2):
CatBoost ████████████████████████ ~35% AUC 0.94644
HistGBM ████████████████████ ~30% AUC 0.94510
LightGBM ████████████ ~20% AUC 0.94200
XGBoost ██████ ~10% AUC 0.87500
LogReg ████ ~5% AUC 0.87000
Stage 1: Find optimal weights for models with OOF data via scipy SLSQP
Minimise: -AUC( Σ w_i × OOF_i )
Subject to: Σ w_i = 1, w_i ≥ 0
Result:
CatBoost 0.6xxx
HistGBM 0.3xxx
LogReg 0.000 ← optimizer found it hurts the blend
OOF blend AUC = 0.94673 (better than any single model)
Stage 2: Combine OOF-optimised meta-model with remaining models
using AUC^3 power weighting
oof_blend ████████████████████████████ ~80%
LightGBM ████████ ~15%
XGBoost ████ ~5%
┌────────────┐
OOF preds ──▶ │ CatBoost │──┐
└────────────┘ │
┌────────────┐ │ ┌──────────────────┐
OOF preds ──▶ │ HistGBM │──┼──▶│ LightGBM │──▶ Final
└────────────┘ │ │ Meta-Learner │ Prediction
┌────────────┐ │ │ (shallow, 5-fold)│
OOF preds ──▶ │ LightGBM │──┘ └──────────────────┘
└────────────┘
OOF preds ──▶ │ LogReg │
└────────────┘
Meta-learner: LightGBM with constrained hyperparameters (num_leaves=15, max_depth=4) to prevent overfitting on meta-features.
┌──────────────────────────────────────────────────────────────┐
│ OOF ROC-AUC Leaderboard │
├─────────────────────────────┬────────────────────────────────┤
│ CatBoost │ 0.94644 ████████████████████ │
│ HistGBM │ 0.94510 ████████████████████ │
│ OOF-Optimised Blend (v3) │ 0.94673 ████████████████████ │
│ LightGBM │ 0.94200 ███████████████████ │
│ XGBoost │ 0.87500 ████████████████ │
│ BiLSTM │ 0.81000 ███████████████ │
│ Logistic Regression │ ~0.87 ████████████████ │
└─────────────────────────────┴────────────────────────────────┘
✅ TyreLife is the single strongest predictor of pit stop timing
✅ Pit window distance features (Wave 2 FE) gave a measurable AUC lift
✅ CatBoost's ordered boosting generalises better than XGBoost on this data
✅ Scipy-optimised blending outperformed equal-weight averaging
✅ BiLSTM struggled without access to clean full-sequence race data
Pit probability histogram (test set):
0.0–0.1 ████████████████████████████████████████ most laps — no pit
0.1–0.2 ████
0.2–0.3 ███
0.3–0.4 ██
0.4–0.5 ██
0.5–0.6 ██
0.6–0.7 ██
0.7–0.8 ███
0.8–0.9 ████
0.9–1.0 █████████████████ clear pit laps
The bimodal distribution confirms the model is confidently separating pit from non-pit laps, rather than hedging near 0.5.
Predicting F1 Pit Stops/
│
├── Data/
│ ├── train.csv ← Raw training data
│ ├── test.csv ← Raw test data
│ ├── sample_submission.csv ← Submission format
│ ├── train_fe.parquet ← Wave 1+2 engineered features (train)
│ ├── test_fe.parquet ← Wave 1+2 engineered features (test)
│ ├── train_fe2.parquet ← + Pit window features (train)
│ └── test_fe2.parquet ← + Pit window features (test)
│
├── Notebooks/
│ ├── eda.ipynb ← Exploratory data analysis
│ ├── feature_engineering.ipynb ← Feature engineering walkthrough
│ ├── xgboost_pitstop.ipynb ← XGBoost experiments
│ ├── lgbm_pitstop.ipynb ← LightGBM experiments
│ └── bilstm_pitstop.ipynb ← BiLSTM sequence model
│
├── Models/
│ ├── catboost_pitstop.py ← CatBoost (best single model, AUC 0.946)
│ ├── histgbm_pitstop.py ← HistGradientBoosting (AUC 0.945)
│ ├── randomforest_pitstop.py ← Random Forest baseline
│ ├── logreg_pitstop.py ← Logistic Regression (diversity)
│ ├── ridge_pitstop.py ← Ridge classifier
│ ├── optuna_catboost.py ← Optuna hyperparameter tuning (CatBoost)
│ ├── optuna_lgbm.py ← Optuna hyperparameter tuning (LGBM)
│ └── advanced_pitstop.py ← LGBM + CatBoost with domain features
│
├── Feature Engineering/
│ ├── feature_engineering_v2.py ← Pit window distance features (Wave 2)
│ └── feature_engineering_executed.ipynb
│
├── Ensemble/
│ ├── gen_submission.py ← v1 simple weighted blend
│ ├── gen_final_submission.py ← v1 AUC-weighted blend
│ ├── gen_final_submission_v2.py ← v2 rank-avg + AUC^5
│ ├── gen_final_submission_v3.py ← v3 scipy-optimised (final)
│ └── stacking_ensemble.py ← Level-2 LightGBM stacking
│
├── Saved OOF predictions/
│ ├── catboost_oof_preds.npy
│ ├── histgbm_oof_preds.npy
│ ├── logreg_oof_preds.npy
│ └── ridge_oof_preds.npy
│
├── Evaluation plots/
│ ├── catboost_evaluation.png
│ ├── catboost_feature_importance.png
│ ├── lgbm_evaluation.png
│ ├── lgbm_feature_importance.png
│ ├── logreg_evaluation.png
│ └── histgbm_evaluation.png
│
└── Submissions/
├── submission_catboost.csv
├── submission_histgbm.csv
├── submission_lgbm.csv
├── submission_xgboost.csv
├── submission_bilstm.csv
├── submission_logreg.csv
├── submission_final.csv ← v1
├── submission_final_v2.csv ← v2
└── submission_final_v3.csv ← v3 (submitted)
pip install lightgbm catboost xgboost scikit-learn optuna scipy pandas numpy matplotlib seaborn pyarrow# 1. Feature engineering
python feature_engineering_v2.py # adds pit window features
# 2. Train individual models
python catboost_pitstop.py # AUC 0.946 — run this first
python histgbm_pitstop.py # AUC 0.945
python logreg_pitstop.py # diversity model
# 3. Generate final ensemble submission
python gen_final_submission_v3.py # scipy-optimised blend| What | Value |
|---|---|
| Task | Binary classification (PitNextLap) |
| Metric | ROC-AUC |
| Best single model | CatBoost — 0.94644 |
| Best ensemble OOF | Scipy-optimised blend — 0.94673 |
| Class imbalance | ~7:1 (no pit : pit) |
| CV strategy | 5-fold Stratified |
| Top feature | TyreLife |
| Final submission | submission_final_v3.csv |
Built with LightGBM · CatBoost · XGBoost · scikit-learn · Optuna · scipy