---

PyPI | Documentation | API | Changelog | Examples | Releases | Docker | Binder

A toolbox for the calibration and evaluation of simulation models.

Table of contents

• Table of contents
• Introduction
• Features and Functionality
• Quickstart
• Installation
• Usage with Docker
• Usage with Binder
• Communication
• Contributions and Support
• License
• Funding and Acknowledgements

Introduction

calisim is an open-source, low-code model calibration library that streamlines and standardises your workflows, while aiming to be as flexible and extensible as needed to support more complex use-cases. Using calisim will speed up your experiment cycle substantially and make you more productive.

calisim is primarily a wrapper around popular libraries and frameworks including Optuna, PyMC, scikit-learn, and emcee among many others. The design and simplicity of calisim was inspired by the scikit-learn and PyCaret libraries.

Features and Functionality

• A standardised and streamlined interface to multiple calibration procedures and libraries.
• A low-code library, allowing modellers to rapidly construct multiple workflows for many calibration procedures.
• An object-oriented programming architecture, allowing users to easily extend and modify calibration workflows for their own complex modelling use-cases.
• An unopinionated approach to working with simulation models, allowing users to calibrate both Python-based and non-Python-based models.
• Optional integration with PyTorch for access to more sophisticated Gaussian Process and deep learning surrogate models, state-of-the-art evolutionary algorithms, and deep generative modelling for simulation-based inference.

Quickstart

# Load imports
import numpy as np
import pandas as pd

from calisim.data_model import (
	DistributionModel,
	ParameterDataType,
	ParameterSpecification,
)
from calisim.example_models import LotkaVolterraModel
from calisim.optimisation import OptimisationMethod, OptimisationMethodModel
from calisim.statistics import MeanSquaredError
from calisim.utils import get_examples_outdir

# Get model
model = LotkaVolterraModel()
observed_data = model.get_observed_data()

# Specify model parameter distributions
parameter_spec = ParameterSpecification(
	parameters=[
		DistributionModel(
			name="alpha",
			distribution_name="uniform",
			distribution_args=[0.45, 0.55],
			data_type=ParameterDataType.CONTINUOUS,
		)
	]
)

# Define objective function
def objective(
	parameters: dict, simulation_id: str, observed_data: np.ndarray | None, t: pd.Series
) -> float | list[float]:
	simulation_parameters = dict(
		alpha=parameters["alpha"],
		beta=0.024, h0=34.0, l0=5.9,
		t=t, gamma=0.84, delta=0.026,
	)

	simulated_data = model.simulate(simulation_parameters).lynx.values
	metric = MeanSquaredError()
	discrepancy = metric.calculate(observed_data, simulated_data)
	return discrepancy

# Specify calibration parameter values
specification = OptimisationMethodModel(
	experiment_name="optuna_optimisation",
	parameter_spec=parameter_spec,
	observed_data=observed_data.lynx.values,
	outdir=get_examples_outdir(),
	method="tpes",
	directions=["minimize"],
	n_iterations=100,
	method_kwargs=dict(n_startup_trials=50),
	calibration_func_kwargs=dict(t=observed_data.year),
)

# Choose calibration engine
calibrator = OptimisationMethod(
	calibration_func=objective, specification=specification, engine="optuna"
)

# Run the workflow
calibrator.specify().execute().analyze()

# View the results
result_artifacts = "\n".join(calibrator.get_artifacts())
print(f"View results: \n{result_artifacts}")
print(f"Parameter estimates: {calibrator.get_parameter_estimates()}")

Installation

The easiest way to install calisim is by using pip:

pip install calisim

calisim's default installation will not include all optional dependencies. You may be interested in one or more extras:

# Install PyTorch extras
pip install calisim[torch]

# Install Hydra extras
pip install calisim[hydra]

# Install TorchX extras
pip install calisim[torchx]

# Install multiple extras
pip install calisim[torch,hydra,torchx]

Usage with Docker

You may also want to execute calisim inside of a Docker container. You can do so by running the following:

# Change the image version as needed
export CALISIM_VERSION=latest

# Get docker-compose.yaml file
wget https://raw.githubusercontent.com/Plant-Food-Research-Open/calisim/refs/heads/main/docker-compose.yaml

# Pull the image
docker compose pull calisim

# Run an example
docker compose run --rm calisim python examples/optimisation/optuna_example.py

# Pull and run the image with PyTorch dependencies included
# docker compose pull calisim_torch
# docker compose run --rm calisim_torch python examples/optimisation/botorch_example.py

Usage with Binder

You may wish to work with calisim using MyBinder. You can do so by accessing the following link.

Communication

Please refer to the following links:

• GitHub Discussions for questions.
• GitHub Issues for bug reports and feature requests.

Contributions and Support

Contributions are more than welcome. For general guidelines on how to contribute to this project, take a look at CONTRIBUTING.md.

For our community code of conduct, please also view CODE_OF_CONDUCT.md.

License

calisim is published under the Apache License (see LICENSE).

View all third party licenses (see third_party)

Funding and Acknowledgements

This work was funded by the New Zealand Ministry of Business, Innovation, and Employment (MBIE) through the Strategic Science Investment Fund (contract number C11X1702) as part of the Digital Horticultural Systems Ngā Pou Rangahau, a Growing Futures™ programme at Plant & Food Research. We sincerely thank the Fruit Physiology Group, orchard managers, and all those who contributed their time and effort to the management and operation of the orchards. Your dedication has been instrumental in the success of this project. Our gratitude extends to the wider Digital Horticultural Systems team for their invaluable support in the modelling and software development process. Our work would not have been possible without the considerable dedication and collaboration of all parties involved.