r-laminr on conda-forge

[ap--]

Hello everyone 👋

If laminr would be available on conda-forge (or some other conda-channel) it would simplify shipping working lamin conda-environments bundling R and Python dependencies via pixi.

I played around with this and managed to install a complete working conda env (with jupyter and lamindb and laminr) from (basically) one pixi.toml file (and ship the entire env as a relocatable packaged conda env). It seems that pixi-pack might get s3 support: Quantco/pixi-pack#91 which would then enable deploying fully working conda-envs (with cross OS support) with python or python&R by providing a single s3 uri to pixi-pack.

This is currently very crude and verbose, but here's an example POC:

[project]
channels = ["conda-forge", "bioconda"]
name = "jupyter-lamin"
platforms = ["linux-64"]

[tasks]
start = "jupyter lab"

[dependencies]
# python deps (should move as many upstream deps here)
#   (lamindb is not available on conda, but also doesn't have to)
python = "3.12.*"
ipython = "*"
ipywidgets = "*"
jupyterlab = "*"
scipy = "<1.15.0"
urllib3 = "<2"
zarr = "<3"
universal_pathlib = "==0.2.6"
fsspec = ">2024.10.0,<=2025.2.0"
pip = "*"
# R dependencies all from conda
r-base = ">=4.4.0"
r-irkernel = ">=1"
r-cli = "*"
r-purrr = "*"
"r-r.utils" = "*"
r-r6 = "*"
r-reticulate = "*"
r-rlang = "*"
r-withr = "*"
r-anndata = "*"  # not avialable on osx64 for R>=4.4
r-arrow = "*"
r-jsonlite = "*"
r-knitr = "*"
r-nanoparquet = "*"
r-quarto = "*"
r-readr = "*"
r-rstudioapi = "*"
r-rsvg = "*"
r-seurat = "*"
r-testthat = "*"
r-yaml = "*"

[pypi-dependencies]
lamindb = { version = ">=1.3", extras = ["bionty", "gcp", "jupyter", "wetlab", "zarr"] }

[activation]
scripts = [".pixi_support/fix_reticulate.sh", ".pixi_support/ensure_laminr.sh"]

[activation.env]
PIXI_R_LIBS = "$CONDA_PREFIX/lib/R/library"
R_LIBS = "$PIXI_R_LIBS"
R_LIBS_USER = "$PIXI_R_LIBS"
RETICULATE_PYTHON= "$CONDA_PREFIX/bin/python"  # needed to point reticulate to the conda python

In this example there are currently two support scripts that work around issues with this.

1. .pixi_support/fix_reticulate.sh is due to Support for pixi environments? rstudio/reticulate#1650 and basically applies the conda history fix suggested in this issue
2. .pixi_support/ensure_laminr.sh just checks if laminr is installed in the R enviroment and installs it if not (because you can't afaik have cran dependencies in conda environment.ymls)

#!/bin/bash
# .pixi_support/ensure_laminr.sh

# pixi install does not support postrun hooks, which is what we'd need.
# that or a r-laminr conda-forge package

$CONDA_PREFIX/bin/Rscript -e "library(laminr)" > /dev/null && exit_status=$? || exit_status=$? ; true

if [ $exit_status -ne 0 ]; then
        echo "INSTALLING LAMINR"
        $CONDA_PREFIX/bin/Rscript -e 'install.packages("laminr", dependencies=TRUE, repos="http://cran.us.r-project.org")'
else
        echo "laminr is installed"
fi

#!/bin/bash
# .pixi_support/fix_reticulate.sh

echo -e "# cmd: $CONDA_PREFIX/bin/conda" > $CONDA_PREFIX/conda-meta/history

[lazappi]

I think this would be fairly straightforward and useful to people but it probably needs to be coordinated with lamindb. @falexwolf do you have any plans to add lamindb to conda-forge?

[falexwolf]

Agreed, Luke.

The request came up before but we didn't follow through. There is a longterm plan but it wasn't yet high up on the priority list.

[ap--]

I don't think it would require lamindb (and it's laminlabs python dependencies) to be available on conda, because python dependencies on pypi can still be specified via a conda environment.yml.

So it's really only laminr (all its R dependencies seem to be available via bioconda and conda-forge)

[ap--]

Actually, in the time I spent writing this issue, I could have probably created the feedstock PR in conda-forge...

If that is okay with you, I'll set it up when I have a bit of idle time and add you as feedstock maintainers.

[falexwolf]

It's definitely ok, Andreas! Thank you so much! 🙏

[ap--]

The feedstock staging PR is pushed: conda-forge/staged-recipes#29719

In the PR we need a comment from @falexwolf, @rcannood and @lazappi to agree that you're willing to be feedstock maintainers, and then we should be good to label for review ☺

[ap--]

Btw, I don't know much about R packaging, but I noticed that the Apache-2.0 license is not packaged in the laminr tarball. I don't know if cran requires anything like that, just letting you know in case it's missing by accident.

[lazappi]

Btw, I don't know much about R packaging, but I noticed that the Apache-2.0 license is not packaged in the laminr tarball. I don't know if cran requires anything like that, just letting you know in case it's missing by accident.

I think for standard licenses CRAN doesn't want the file included in the build tarball, the name of the license is just included in DESCRIPTION.

[ap--]

So conda-forge/staged-recipes#29719 got merged and https://github.com/conda-forge/r-laminr-feedstock is now available. This is pretty cool, because now a pixi.toml can be provided to allow users on all operating systems to run a single command to setup a fully working jupyter lab environment with Python and R kernels for lamindb and laminr.

pixi.toml

[project]
channels = ["conda-forge", "bioconda"]
name = "jupyterlab-lamin"
platforms = ["linux-64", "osx-arm64", "osx-64", "win-64"]

[system-requirements]
macos = "14.0"

[tasks._fix_reticulate]
# workaround for https://github.com/rstudio/reticulate/issues/1650
cmd = 'echo "# cmd: $CONDA_PREFIX/bin/conda" > $CONDA_PREFIX/conda-meta/history'

[tasks]
jupyter = { cmd = "jupyter lab", depends-on = ["_fix_reticulate"] }

[dependencies]
python = "3.12.*"
urllib3 = "<2"  # workaround for all the interdependent pinning in lamin packages
pip = "*"
# laminr
r-base = ">=4.4.0"
r-laminr = "*"
r-anndata = { version="*", channel="bioconda" }
r-arrow = "*"
r-jsonlite = "*"
r-knitr = "*"
r-nanoparquet = "*"
r-quarto = "*"
r-readr = "*"
r-rstudioapi = "*"
r-rsvg = "*"
r-seurat = "*"
r-testthat = "*"
r-yaml = "*"
# jupyter
ipython = "*"
ipywidgets = "*"
jupyterlab = "*"
r-irkernel = ">=1"

[pypi-dependencies]
lamindb = { version = ">=1.6.0", extras = ["bionty", "clinicore", "gcp", "jupyter", "wetlab", "zarr"] }

[activation.env]
PIXI_R_LIBS = "$CONDA_PREFIX/lib/R/library"
R_LIBS = "$PIXI_R_LIBS"
R_LIBS_USER = "$PIXI_R_LIBS"
RETICULATE_PYTHON= "$CONDA_PREFIX/bin/python"

Copying this file to a folder on your machine, and running pixi run jupyter in the same folder will download all required packages and launches jupyter lab in the correct env.