PyBandits

PyBandits is a Python library for Multi-Armed Bandit. It provides an implementation of stochastic Multi-Armed Bandit (sMAB) and contextual Multi-Armed Bandit (cMAB) based on Thompson Sampling.

For the sMAB, we implemented a Bernoulli multi-armed bandit based on Thompson Sampling algorithm Agrawal and Goyal, 2012. If context information is available we provide a generalisation of Thompson Sampling for cMAB Agrawal and Goyal, 2014 implemented with PyMC3, an open source probabilistic programming framework for automatic Bayesian inference on user-defined probabilistic models.

Installation

This library is distributed on PyPI and can be installed with pip.

pip install pybandits

Based on the guidelines of pymc authors, it is highly recommended to install the library in a conda environment via the following.

conda install -c conda-forge pymc
pip install pybandits

The command above will automatically install all the dependencies listed in pyproject.toml. Please visit the installation page for more details.

Getting started

A short example, illustrating it use. Use the sMAB model to predict actions and update the model based on rewards from the environment.

import numpy as np
from pybandits.model import Beta
from pybandits.smab import SmabBernoulli

n_samples=100

# define action model
actions = {
    "a1": Beta(),
    "a2": Beta(),
}

# init stochastic Multi-Armed Bandit model
smab = SmabBernoulli(actions=actions)

# predict actions
pred_actions, _ = smab.predict(n_samples=n_samples)
simulated_rewards = np.random.randint(2, size=n_samples)

# update model
smab.update(actions=pred_actions, rewards=simulated_rewards)

Documentation

For more information please read the full documentation and tutorials.

Info for developers

The source code of the project is available on GitHub.

git clone https://github.com/playtikaoss/pybandits.git

You can install the library and the dependencies from the source code with one of the following commands:

poetry install                # install library + dependencies
poetry install --without dev     # install library + dependencies, excluding developer-dependencies

To create the HTML documentation run the following commands:

cd docs/src
make html

Run tests

Tests can be executed with pytest running the following commands. Make sure to have the library installed before to run any tests.

cd tests
pytest -vv                                      # run all tests
pytest -vv test_testmodule.py                   # run all tests within a module
pytest -vv test_testmodule.py -k test_testname  # run only 1 test
pytest -vv -k 'not time'                        # run all tests but not exec time

License

MIT License