|
1 | | -## AdaBoost |
| 1 | +# AdaBoost |
2 | 2 |
|
3 | | -Project for [Statistical Methods for Machine Learning](https://cesa-bianchi.di.unimi.it/MSA/index_25-26.html) course at Università degli Studi di Milano - Master degree in Computer Science (a.y. 2025/26). |
| 3 | +Project for the [Statistical Methods for Machine Learning](https://cesa-bianchi.di.unimi.it/MSA/index_25-26.html) course at Università degli Studi di Milano - Master's degree in Computer Science (a.y. 2025/26). |
| 4 | + |
| 5 | +Compiled Report PDF: [report.pdf](./report/report.pdf) |
4 | 6 |
|
5 | 7 | ## Abstract |
6 | 8 |
|
7 | | -TODO |
| 9 | +The project implements the Adaptive Boosting algorithm (**AdaBoost**) from scratch using **decision stumps** (decision trees of depth 1) as weak learners. |
| 10 | +The aim of the study is to understand how boosting combines weak learners, focuses on hard examples, and minimizes the classification error over training rounds. |
| 11 | + |
| 12 | +Experiments are run on one real-world dataset (UCI Adult) and three synthetic datasets (linearly separable, XOR, concentric circles). |
| 13 | +AdaBoost is then compared against a single **decision tree** and **logistic regression**. |
| 14 | +Results show that AdaBoost performs very well on most datasets, but completely fails on the XOR dataset where simple decision stumps cannot beat random guessing. |
8 | 15 |
|
9 | 16 | ## Project Structure |
10 | 17 |
|
11 | | -``` |
12 | | -├── notebook.ipynb # results and graphs |
| 18 | +```text |
| 19 | +├── 01_datasets.ipynb # Dataset loading and preprocessing |
| 20 | +├── 02_hyperparameters.ipynb # Hyperparameters tuning |
| 21 | +├── 03_adaboost.ipynb # AdaBoost analysis |
| 22 | +├── 04_comparison.ipynb # Baseline models comparison |
13 | 23 | ├── src/ |
14 | | -│ ├── __init__.py |
15 | | -│ ├── data.py # dataset |
16 | | -│ └── adaboost.py # adaboost from scratch |
17 | | -├── uv.lock # project metadata |
18 | | -└── pyproject.toml # project metadata |
| 24 | +│ ├── data.py # Data loading and generation |
| 25 | +│ ├── decision_stump.py # Weighted decision stump weak learner |
| 26 | +│ ├── decision_tree.py # Decision tree |
| 27 | +│ ├── logistic_regression.py # Logistic regression |
| 28 | +│ ├── adaboost.py # AdaBoost ensemble class |
| 29 | +│ └── hyperparameters.py # Helper functions for hyperparameter tuning |
| 30 | +├── report/ |
| 31 | +│ ├── report.typ # Source of the report |
| 32 | +│ └── report.pdf # Compiled report PDF |
| 33 | +└── figs/ # Plots and figures |
19 | 34 | ``` |
20 | 35 |
|
21 | | -## Run |
| 36 | +## Setup |
| 37 | + |
| 38 | +Dependencies and virtual environments managed by [uv](https://github.com/astral-sh/uv). |
| 39 | +Formatting and linting using [ruff](https://github.com/astral-sh/ruff) and [ty](https://github.com/astral-sh/ty). |
22 | 40 |
|
23 | | -Dependencies and venv managed by uv _(which is better than pip)_. |
| 41 | +Install dependencies: |
24 | 42 |
|
25 | 43 | ```bash |
26 | 44 | uv sync |
| 45 | +``` |
| 46 | + |
| 47 | +Run the notebooks: |
| 48 | + |
| 49 | +```bash |
27 | 50 | uv run jupyter lab |
28 | 51 | ``` |
0 commit comments