Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 12 additions & 19 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,22 @@ on:
release:
types: [published]

permissions:
id-token: write

jobs:
deploy:
runs-on: ubuntu-latest
environment: pypi

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Install dependencies
run: |
poetry install

- name: Install uv
uses: astral-sh/setup-uv@v3

- name: Build package
run: |
poetry build
- name: Publish package
env:
PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
poetry config pypi-token.pypi $PYPI_TOKEN
poetry publish
run: uv build

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
78 changes: 28 additions & 50 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,61 +16,39 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
python-version: ["3.9", "3.11", "3.12"]
# Full matrix only for PRs to main
os: ${{ github.event_name == 'pull_request' && github.base_ref == 'main' && fromJSON('["ubuntu-latest", "windows-latest"]') || fromJSON('["ubuntu-latest"]') }}
python-version: ${{ github.event_name == 'pull_request' && github.base_ref == 'main' && fromJSON('["3.9", "3.11", "3.12"]') || fromJSON('["3.12"]') }}
defaults:
run:
shell: bash -l {0}

steps:
- name: Set OS and Python version
id: set-vars
run: |
if [[ "${GITHUB_REF}" == "refs/heads/main" || "${GITHUB_REF}" == "refs/heads/dev" ]]; then
echo "os-matrix=ubuntu-latest,windows-latest" >> $GITHUB_ENV
echo "python-matrix=3.9,3.11,3.12" >> $GITHUB_ENV
else
echo "os-matrix=ubuntu-latest" >> $GITHUB_ENV
echo "python-matrix=3.12" >> $GITHUB_ENV
fi
- name: Checkout
uses: actions/checkout@v3
- name: Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Cache Poetry
uses: actions/cache@v3
with:
path: |
~/.cache/pypoetry
~/.cache/pip
key: ${{ runner.os }}-poetry-${{ matrix.python-version }}-${{ hashFiles('**/poetry.lock') }}
restore-keys: |
${{ runner.os }}-poetry-${{ matrix.python-version }}-
- name: Poetry
uses: snok/install-poetry@v1
with:
version: 1.8.3
- name: Lock
run: poetry lock --no-update
- name: Install
run: poetry install
- name: Checkers
run: make checkers
- name: Codecov
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
- name: Check Changed Files
id: changed-files
uses: tj-actions/changed-files@v40
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
files: |
docs/**
**.rst
enable-cache: true
cache-dependency-glob: "uv.lock"

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

- name: Install dependencies
run: make install

- name: Check with ruff
run: make check-quality

- name: Test with pytest
run: make check-tests

- name: Check types
run: make check-types

- name: Build Docs
if: matrix.python-version == '3.12' && matrix.os == 'ubuntu-latest' && steps.changed-files.outputs.any_changed == 'true'
- name: Build and test docs
if: matrix.python-version == '3.12' && matrix.os == 'ubuntu-latest'
run: |
poetry run sphinx-build -b html docs/ _build/html
make doc
make doctest
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ var/
*.egg-info/
.installed.cfg
*.egg
poetry.lock

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
1 change: 0 additions & 1 deletion .python-version
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
3.12.10
9 changes: 3 additions & 6 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,12 @@ Local setup

We encourage you to use a virtual environment. You'll want to activate it every time you want to work on `Qolmat`.

You can create a virtual environment via `conda`:
You can create a virtual environment and install dependencies with `uv`:

.. code:: sh

$ pip install poetry
$ poetry config virtualenvs.in-project true
$ poetry lock
$ poetry install
$ poetry shell
$ uv sync
$ uv run pytest

Once the environment is installed, pre-commit is installed, but need to be activated using the following command:

Expand Down
2 changes: 2 additions & 0 deletions HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ History
0.1.11 (2025-12-30)
------------------
* PLKM test implemented and documented in the holes_characterization module
* Dependency management improved with uv
* Migrated from hyperopt to skopt for hyperparameter optimization

0.1.10 (2025-08-30)
------------------
Expand Down
41 changes: 26 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,34 +1,45 @@
# Install dependencies
install:
uv sync --all-extras

# Code quality checks
check-coverage:
poetry run pytest --cov-branch --cov=qolmat/ --cov-report=xml tests/

check-poetry:
poetry check --lock
uv run pytest --cov-branch --cov=qolmat/ --cov-report=xml tests/

check-quality:
poetry run ruff check qolmat/ tests/
uv run ruff check qolmat/ tests/

check-security:
poetry run bandit --recursive --configfile=pyproject.toml qolmat/
uv run bandit --recursive --configfile=pyproject.toml qolmat/

check-tests:
poetry run pytest tests/
uv run pytest tests/

check-types:
poetry run mypy qolmat/ tests/
uv run mypy qolmat/ tests/

checkers: check-coverage check-types

# Formatting
format:
uv run ruff format qolmat/ tests/
uv run ruff check --fix qolmat/ tests/

# Cleaning
clean:
rm -rf .mypy_cache .pytest_cache .coverage*
rm -rf .mypy_cache .pytest_cache .coverage* .ruff_cache
rm -rf **__pycache__
make clean -C docs

coverage:
poetry run pytest --cov-branch --cov=qolmat --cov-report=xml tests
uv run make clean -C docs

# Documentation
doc:
make html -C docs
uv run make html -C docs

doctest:
poetry run pytest --doctest-modules --pyargs qolmat
uv run pytest --doctest-modules --pyargs qolmat

# Development helpers
lock:
uv lock

.PHONY: install check-coverage check-quality check-security check-tests check-types checkers format clean doc doctest lock
2 changes: 0 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,6 @@
# see https://github.com/numpy/numpydoc/issues/69
numpydoc_show_class_members = False

from distutils.version import LooseVersion

# pngmath / imgmath compatibility layer for different sphinx versions
# import sphinx

Expand Down
18 changes: 9 additions & 9 deletions docs/sg_execution_times.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Computation times
=================
**00:00.000** total execution time for 6 files **from all galleries**:
**01:04.897** total execution time for 6 files **from all galleries**:

.. container::

Expand All @@ -33,20 +33,20 @@ Computation times
- Time
- Mem (MB)
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_benchmark_TS.py` (``../examples/tutorials/plot_tuto_benchmark_TS.py``)
- 00:00.000
- 00:21.381
- 0.0
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_categorical.py` (``../examples/tutorials/plot_tuto_categorical.py``)
- 00:00.000
- 00:16.026
- 0.0
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_diffusion_models.py` (``../examples/tutorials/plot_tuto_diffusion_models.py``)
- 00:00.000
- 0.0
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_hole_generator.py` (``../examples/tutorials/plot_tuto_hole_generator.py``)
- 00:00.000
- 00:11.896
- 0.0
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_mcar.py` (``../examples/tutorials/plot_tuto_mcar.py``)
- 00:00.000
- 00:08.072
- 0.0
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_mean_median.py` (``../examples/tutorials/plot_tuto_mean_median.py``)
- 00:00.000
- 00:04.939
- 0.0
* - :ref:`sphx_glr_examples_tutorials_plot_tuto_hole_generator.py` (``../examples/tutorials/plot_tuto_hole_generator.py``)
- 00:02.583
- 0.0
18 changes: 8 additions & 10 deletions examples/benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ First, import some useful libraries
```python tags=[]
import warnings
# warnings.filterwarnings('error')
```

```python tags=[]
%reload_ext autoreload
%autoreload 2

Expand All @@ -33,21 +31,21 @@ from IPython.display import Image
import pandas as pd
from datetime import datetime
import numpy as np
import hyperopt as ho
from skopt.space import Real, Integer, Categorical
np.random.seed(1234)
from matplotlib import pyplot as plt
import matplotlib.ticker as plticker

tab10 = plt.get_cmap("tab10")
plt.rcParams.update({'font.size': 18})


from sklearn.linear_model import LinearRegression

from qolmat.benchmark import comparator, missing_patterns
from qolmat.imputations import imputers
from qolmat.utils import data, utils, plot


```

### **I. Load data**
Expand Down Expand Up @@ -124,15 +122,15 @@ imputer_residuals = imputers.ImputerResiduals(groups=("station",), period=365, m
imputer_rpca = imputers.ImputerRpcaNoisy(groups=("station",), columnwise=False, max_iterations=500, tau=.01, lam=5, rank=1)
imputer_rpca_opti = imputers.ImputerRpcaNoisy(groups=("station",), columnwise=False, max_iterations=256)
dict_config_opti["RPCA_opti"] = {
"tau": ho.hp.uniform("tau", low=.5, high=5),
"lam": ho.hp.uniform("lam", low=.1, high=1),
"tau": Real(0.5, 5.0, name="tau"),
"lam": Real(0.1, 1.0, name="lam"),
}
imputer_rpca_opticw = imputers.ImputerRpcaNoisy(groups=("station",), columnwise=False, max_iterations=256)
dict_config_opti["RPCA_opticw"] = {
"tau/TEMP": ho.hp.uniform("tau/TEMP", low=.5, high=5),
"tau/PRES": ho.hp.uniform("tau/PRES", low=.5, high=5),
"lam/TEMP": ho.hp.uniform("lam/TEMP", low=.1, high=1),
"lam/PRES": ho.hp.uniform("lam/PRES", low=.1, high=1),
"tau/TEMP": Real(0.5, 5.0, name="tau/TEMP"),
"tau/PRES": Real(0.5, 5.0, name="tau/PRES"),
"lam/TEMP": Real(0.1, 1.0, name="lam/TEMP"),
"lam/PRES": Real(0.1, 1.0, name="lam/PRES"),
}

imputer_normal_sample = imputers.ImputerEM(groups=("station",), model="multinormal", method="sample", max_iter_em=8, n_iter_ou=128, dt=4e-2)
Expand Down
Loading
Loading