Multi-level Monte-Carlo (MLMC) and various reinforcement learning algorithms

Reinforcement learning is wrong on so many levels.

This repositoru contains code for the paper "Methods for Optimization Problems with Markovian Stochasticity and Non-Euclidean Geometry" and some other snippets that the authors found useful.

Installation

The environment is built in docker, you can build the container:

make build

And run it in detached mode

make run

Also, you can just install the requirements for the project with pip, uv, or conda.

Experiment scripts

The experiments for the paper are located in the paper_experiments. Each script is configured with a hydra config, entitled the same as the experiment file.

Experiments with projections:

cd paper_experiments;
python experiment_mdpo.py

Experiments without projections:

cd paper_experiments;
python experiment_ampo.py

Note

To enable wandb logging, don't forget to specify the WANDB_API_KEY environment variable.