Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 21 additions & 58 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,47 +13,19 @@ concurrency:
cancel-in-progress: true

jobs:
# Get the test environment from hatch as defined in pyproject.toml.
# This ensures that the pyproject.toml is the single point of truth for test definitions and the same tests are
# run locally and on continuous integration.
# Check [[tool.hatch.envs.hatch-test.matrix]] in pyproject.toml and https://hatch.pypa.io/latest/environment/ for
# more details.
get-environments:
runs-on: ubuntu-latest
outputs:
envs: ${{ steps.get-envs.outputs.envs }}
steps:
- uses: actions/checkout@v4
with:
filter: blob:none
fetch-depth: 0
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Get test environments
id: get-envs
run: |
ENVS_JSON=$(uvx hatch env show --json | jq -c 'to_entries
| map(
select(.key | startswith("hatch-test"))
| {
name: .key,
label: (if (.key | contains("pre")) then .key + " (PRE-RELEASE DEPENDENCIES)" else .key end),
python: .value.python
}
)')
echo "envs=${ENVS_JSON}" | tee $GITHUB_OUTPUT

# Run tests through hatch. Spawns a separate runner for each environment defined in the hatch matrix obtained above.
test:
needs: get-environments

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
env: ${{ fromJSON(needs.get-environments.outputs.envs) }}

name: ${{ matrix.env.label }}
python-version: ["3.12", "3.13"]
prerelease: [false]
include:
- python-version: "3.13"
prerelease: true
os: ubuntu-latest

name: Python ${{ matrix.python-version }}${{ matrix.prerelease && ' (pre-release)' || '' }}
runs-on: ${{ matrix.os }}

steps:
Expand All @@ -64,35 +36,26 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.env.python }}
cache-dependency-glob: pyproject.toml
- name: create hatch environment
run: uvx hatch env create ${{ matrix.env.name }}
- name: run tests using hatch
python-version: ${{ matrix.python-version }}
cache-dependency-glob: uv.lock
- name: Install dependencies
run: uv sync --group test
env:
UV_PRERELEASE: ${{ matrix.prerelease && 'allow' || 'disallow' }}
- name: Run tests
env:
MPLBACKEND: agg
PLATFORM: ${{ matrix.os }}
DISPLAY: :42
run: uvx hatch run ${{ matrix.env.name }}:run-cov -v --color=yes -n auto
- name: generate coverage report
run: |
# See https://coverage.readthedocs.io/en/latest/config.html#run-patch
test -f .coverage || uvx hatch run ${{ matrix.env.name }}:cov-combine
uvx hatch run ${{ matrix.env.name }}:cov-report # report visibly
uvx hatch run ${{ matrix.env.name }}:coverage xml # create report for upload
run: uv run pytest --cov --cov-report=xml -v
- name: Upload coverage
uses: codecov/codecov-action@v5

# Check that all tests defined above pass. This makes it easy to set a single "required" test in branch
# protection instead of having to update it frequently. See https://github.com/re-actors/alls-green#why.
check:
name: Tests pass in all hatch environments
name: Tests pass
if: always()
needs:
- get-environments
- test
needs: [test]
runs-on: ubuntu-latest
steps:
- uses: re-actors/alls-green@release/v1
with:
jobs: ${{ toJSON(needs) }}
- uses: re-actors/alls-green@release/v1
with:
jobs: ${{ toJSON(needs) }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,6 @@ __pycache__/
# docs
/docs/generated/
/docs/_build/
/docs/notebooks/data/
/docs/notebooks/totalvi_test.ipynb
/docs/notebooks/*.h5ad
3 changes: 2 additions & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ build:
- asdf plugin add uv
- asdf install uv latest
- asdf global uv latest
- uv sync --group doc
build:
html:
- uvx hatch run docs:build
- uv run sphinx-build -M html docs docs/_build -W
- mv docs/_build $READTHEDOCS_OUTPUT
24 changes: 19 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,28 @@ and this project adheres to [Semantic Versioning][].
[keep a changelog]: https://keepachangelog.com/en/1.0.0/
[semantic versioning]: https://semver.org/spec/v2.0.0.html

## [0.1.0] - TBA
## [1.0.0] - 2026-06-14

### Fixed

- `MultiVAE(losses=None)` no longer raises `TypeError: argument of type 'NoneType' is not iterable`. `losses=None` (the documented default, meaning MSE for all modalities) now works correctly in both the model wrapper (`_multivae.py`) and the torch module (`_multivae_torch.py`).
- `MultiVAE.load_query_data` no longer raises `AttributeError` when the reference model was trained with `integrate_on` but without an nb/zinb loss (i.e. `theta` is `None`).
- `StratifiedSampler` (used when `integrate_on` is set) no longer silently produces zero training batches on small datasets. Training will now always produce at least one batch per group, and raises a clear `ValueError` if the batch size is too large for the data.
- `organize_multimodal_anndatas` now preserves `.var` metadata from each modality. When the same column name appears in multiple modalities it is suffixed with the modality index (e.g. `mean_0`, `mean_1`). Fixes [#4](https://github.com/theislab/multigrate/issues/4).
- Defensive `.get("columns", [])` in the continuous-covariate registry lookup prevents a `KeyError` with certain scvi-tools versions. Fixes [#45](https://github.com/theislab/multigrate/issues/45).
- Removed deprecated `dtype=` argument from `anndata.AnnData()` call (anndata 0.12+).

### Changed

- Requires Python ≥ 3.12 (aligned with scvi-tools 1.4.3 minimum).
- Bumped core dependencies: `scvi-tools>=1.4.3`, `anndata>=0.12`; removed `numpy<2` and `jax<0.6` upper-bound pins.
- Migrated developer tooling from hatch to native [uv](https://docs.astral.sh/uv/). Dependency groups (`dev`, `test`, `doc`) are now declared via PEP 735 `[dependency-groups]` in `pyproject.toml`.
- CI rewritten to use `uv sync --group test` + `uv run pytest` against Python 3.12 and 3.13.
- ReadTheDocs build updated to use `uv sync --group doc` + `uv run sphinx-build`.
- How the masks for missing modalities are calculated: now assume that the modality is missing when all features are zero instead of the sum over all features being <= 0 previously (only makes a difference when the input can be negative, which doesn't happen with standard single-cell modalities)

### Added

- support for scvi-tools >= 1.0 ([#37](https://github.com/theislab/multigrate/pull/37))
- support for multiple NB lossed ([#34](https://github.com/theislab/multigrate/pull/34))
- tests

### Changed

- how the masks for missing modalities are calculated: now assume that the modality is missing when all features are zero instead of the sum over all features being <= 0 previously (only makes a difference when the input can be negative, which doesn't happen with standard single-cell modalities)
12 changes: 12 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,15 @@

module.MultiVAETorch
```

## Data

```{eval-rst}
.. module:: multigrate.data
.. currentmodule:: multigrate

.. autosummary::
:toctree: generated

data.organize_multimodal_anndatas
```
145 changes: 28 additions & 117 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,128 +11,57 @@ the [scientific Python tutorials][], or the [scanpy developer guide][].
[scientific Python tutorials]: https://learn.scientific-python.org/development/tutorials/
[scanpy developer guide]: https://scanpy.readthedocs.io/en/latest/dev/index.html

:::{tip} The *hatch* project manager
:::{tip} The *uv* project manager

We highly recommend to familiarize yourself with [`hatch`][hatch].
Hatch is a Python project manager that
We recommend using [`uv`][uv] to manage dependencies for this project.
`uv` is a fast Python package manager that

- manages virtual environments, separately for development, testing and building the documentation.
Separating the environments is useful to avoid dependency conflicts.
- allows to run tests locally in different environments (e.g. different python versions)
- allows to run tasks defined in `pyproject.toml`, e.g. to build documentation.

While the project is setup with `hatch` in mind,
it is still possible to use different tools to manage dependencies, such as `uv` or `pip`.
- manages virtual environments and lockfiles (`uv.lock`) for reproducible installs.
- supports dependency groups (dev, test, doc) defined in `pyproject.toml`.
- is used by this project's CI on GitHub Actions.

:::

[hatch]: https://hatch.pypa.io/latest/
[uv]: https://docs.astral.sh/uv/

## Installing dev dependencies

In addition to the packages needed to _use_ this package,
you need additional python packages to [run tests](#writing-tests) and [build the documentation](#docs-building).

:::::{tabs}
::::{group-tab} Hatch

On the command line, you typically interact with hatch through its command line interface (CLI).
Running one of the following commands will automatically resolve the environments for testing and
building the documentation in the background:

```bash
hatch test # defined in the table [tool.hatch.envs.hatch-test] in pyproject.toml
hatch run docs:build # defined in the table [tool.hatch.envs.docs]
```

When using an IDE such as VS Code,
you’ll have to point the editor at the paths to the virtual environments manually.
The environment you typically want to use as your main development environment is the `hatch-test`
environment with the latest Python version.

To get a list of all environments for your projects, run

```bash
hatch env show -i
```

This will list “Standalone” environments and a table of “Matrix” environments like the following:

```
+------------+---------+--------------------------+----------+---------------------------------+-------------+
| Name | Type | Envs | Features | Dependencies | Scripts |
+------------+---------+--------------------------+----------+---------------------------------+-------------+
| hatch-test | virtual | hatch-test.py3.10-stable | dev | coverage-enable-subprocess==1.0 | cov-combine |
| | | hatch-test.py3.13-stable | test | coverage[toml]~=7.4 | cov-report |
| | | hatch-test.py3.13-pre | | pytest-mock~=3.12 | run |
| | | | | pytest-randomly~=3.15 | run-cov |
| | | | | pytest-rerunfailures~=14.0 | |
| | | | | pytest-xdist[psutil]~=3.5 | |
| | | | | pytest~=8.1 | |
+------------+---------+--------------------------+----------+---------------------------------+-------------+
```

From the `Envs` column, select the environment name you want to use for development.
In this example, it would be `hatch-test.py3.13-stable`.

Next, create the environment with

```bash
hatch env create hatch-test.py3.13-stable
```
::::{group-tab} uv

Then, obtain the path to the environment using
To initialise a virtual environment with all development dependencies, run:

```bash
hatch env find hatch-test.py3.13-stable
uv sync --group dev --group test --group doc
```

In case you are using VScode, now open the command palette (Ctrl+Shift+P) and search for `Python: Select Interpreter`.
Choose `Enter Interpreter Path` and paste the path to the virtual environment from above.

In this future, this may become easier through a hatch vscode extension.

::::

::::{group-tab} uv

A popular choice for managing virtual environments is [uv][].
The main disadvantage compared to hatch is that it supports only a single environment per project at a time,
which requires you to mix the dependencies for running tests and building docs.
This can have undesired side-effects,
such as requiring to install a lower version of a library your project depends on,
only because an outdated sphinx plugin pins an older version.

To initalize a virtual environment in the `.venv` directory of your project, simply run
For a minimal environment to run tests only:

```bash
uv sync --all-extras
uv sync --group test
```

The `.venv` directory is typically automatically discovered by IDEs such as VS Code.
The `.venv` directory is automatically discovered by IDEs such as VS Code.

::::

::::{group-tab} Pip

Pip is nowadays mostly superseded by environment manager such as [hatch][].
However, for the sake of completeness, and since it’s ubiquitously available,
we describe how you can manage environments manually using `pip`:

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,test,doc]"
pip install -e “.[tutorials]”
pip install coverage pytest pytest-cov
```

The `.venv` directory is typically automatically discovered by IDEs such as VS Code.

::::
:::::

[hatch environments]: https://hatch.pypa.io/latest/tutorials/environment/basic-usage/
[uv]: https://docs.astral.sh/uv/

## Code-style

This package uses [pre-commit][] to enforce consistent code-styles.
Expand Down Expand Up @@ -175,26 +104,11 @@ This package uses [pytest][] for automated testing.
Please write {doc}`scanpy:dev/testing` for every function added to the package.

Most IDEs integrate with pytest and provide a GUI to run tests.
Just point yours to one of the environments returned by

```bash
hatch env create hatch-test # create test environments for all supported versions
hatch env find hatch-test # list all possible test environment paths
```
Point VS Code at the `.venv` created by `uv sync --group test`.

Alternatively, you can run all tests from the command line by executing
Run all tests from the command line:

:::::{tabs}
::::{group-tab} Hatch

```bash
hatch test # test with the highest supported Python version
# or
hatch test --all # test with all supported Python versions
```

::::

::::{group-tab} uv

```bash
Expand Down Expand Up @@ -227,9 +141,9 @@ The purpose of this check is to detect incompatibilities of new package versions
gives you time to fix the issue or reach out to the developers of the dependency before the package
is released to a wider audience.

The CI job is defined in `.github/workflows/test.yaml`,
however the single point of truth for CI jobs is the Hatch test matrix defined in `pyproject.toml`.
This means that local testing via hatch and remote testing on CI tests against the same python versions and uses the same environments.
The CI job is defined in `.github/workflows/test.yaml`.
It runs `uv sync --group test` and `uv run pytest --cov` against Python 3.12 and 3.13.
Local testing with `uv run pytest` mirrors CI exactly.

## Publishing a release

Expand Down Expand Up @@ -297,22 +211,20 @@ please check out [this feature request][issue-render-notebooks] in the `cookiecu

### Building the docs locally

:::::{tabs}
::::{group-tab} Hatch
First install the doc dependencies:

```bash
hatch run docs:build
hatch run docs:open
uv sync --group doc
```

::::
Then build:

:::::{tabs}
::::{group-tab} uv

```bash
cd docs
uv run sphinx-build -M html . _build -W
(xdg-)open _build/html/index.html
uv run sphinx-build -M html docs docs/_build -W
open docs/_build/html/index.html
```

::::
Expand All @@ -321,9 +233,8 @@ uv run sphinx-build -M html . _build -W

```bash
source .venv/bin/activate
cd docs
sphinx-build -M html . _build -W
(xdg-)open _build/html/index.html
sphinx-build -M html docs docs/_build -W
open docs/_build/html/index.html
```

::::
Expand Down
Loading
Loading