Online Loss Function Learning

This repository contains code for reproducing the experiments in the paper "Meta-Learning Adaptive Loss Functions" by Christian Raymond, Qi Chen, Bing Xue, and Mengjie Zhang. A PyTorch + Higher implementation of the newly proposed Adaptive Loss Function Learning (AdaLFL) algorithm.

Installation

1. Clone this repository to your local machine:

git clone https://github.com/Decadz/Online-Loss-Function-Learning.git
cd Online-Loss-Function-Learning

2. Install the necessary libraries and dependencies:

pip install requirements.txt

Usage

To run the baseline experiments using cross-entropy/squared loss run the following command via the terminal:

python experiments/run_baseline.py --dataset dataset_name --model model_name --seeds [seeds] --device device

To run the offline loss function learning experiments run the following command via the terminal:

python experiments/run_offline.py --dataset dataset_name --model model_name --seeds [seeds] --device device

To run the online loss function learning (AdaLFL) experiments run the following command via the terminal:

python experiments/run_online.py --dataset dataset_name --model model_name --seeds [seeds] --device device

Available Arguments:

• Dataset = {mnist, cifar10, cifar100, svhn, california, boston, diabetes}
• Model = {linear, mlp, lenet5, alexnet, vgg{11, 13, 16, 19}, allcnnc, resnet{18, 34, 50, 101, 152}, preresnet{18, 34, 50, 101, 152}, wrn{40-4, 16-8, 28-10}, squeezenet, pyramidnet}

Code Reproducibility:

The code has not been comprehensively checked and re-run since refactoring. If you're having any issues, find a problem/bug or cannot reproduce similar results as the paper please open an issue or email me.

Reference

If you use our library or find our research of value please consider citing our paper with the following Bibtex entry:

@article{raymond2025adaptive,
  title={Meta-Learning Adaptive Loss Functions},
  author={Raymond, Christian and Chen, Qi and Xue, Bing and Zhang, Mengjie},
  journal={Transactions on Machine Learning Research (TMLR)},
  year={2025}
}
@article{raymond2024thesis,
  title={Meta-Learning Loss Functions for Deep Neural Networks},
  author={Raymond, Christian},
  journal={arXiv preprint arXiv:2406.09713},
  year={2024}
}