nf-dotenv

Automatically source dotenv files into your Nextflow scope.

Visit us at Fulcrum Genomics to learn more about how we can power your Bioinformatics with nf-dotenv and beyond.

Quickstart

Add the plugin to your Nextflow config:

plugins { id 'nf-dotenv' }

And add the following import statement into your processes and workflow files:

include { dotenv } from "plugin/nf-dotenv"

Now you're ready to source environmental file variables in Nextflow contexts!

dotenv("KeyFromEnvironment")

Configuration

This plugin support the following Nextflow configuration:

dotenv.filename = ".env" // Filename of the dotenv file
dotenv.relative = "."    // Relative path of the dotenv file to the main script

A Real World Example

Let's say you have the following Nextflow project:

Dotenv (.env)

SAMTOOLS_VERSION=1.17

Docker Image (Dockerfile)

FROM alpine:3.18

RUN apk add --update --no-cache \
      bash=5.2.15-r5 \
      build-base=0.5-r3 \
      bzip2-dev=1.0.8-r5 \
      xz-dev=5.4.3-r0 \
      zlib-dev=1.2.13-r1

ARG SAMTOOLS_VERSION

RUN wget https://github.com/samtools/samtools/releases/download/${SAMTOOLS_VERSION}/samtools-${SAMTOOLS_VERSION}.tar.bz2 \
    && tar -xjvf samtools-${SAMTOOLS_VERSION}.tar.bz2 \
    && cd samtools-${SAMTOOLS_VERSION} \
    && ./configure --without-curses --enable-configure-htslib \
    && make all all-htslib -j 8 \
    && make install install-htslib \
    && rm -r ../samtools-${SAMTOOLS_VERSION}

Docker Compose (docker-compose.yaml)

services:
  samtools:
    build:
      args:
        SAMTOOLS_VERSION: ${SAMTOOLS_VERSION}
      tags: ['samtools:${SAMTOOLS_VERSION}']

Nextflow Script (main.nf)

include { dotenv } from "plugin/nf-dotenv"

process emit_samtools_version {
    container "samtools:${dotenv('SAMTOOLS_VERSION')}"
    output: stdout

    """
    samtools --version | head -n1
    """
}

workflow { emit_samtools_version() | view }

After building the Docker image with docker compose build, and after enabling Docker for Nextflow, you will be able to use nf-dotenv to source the version tag of the container to use in your Nextflow process. When the main Nextflow script is run with nextflow run main.nf, you will get the following output:

❯ nextflow -quiet run main.nf
samtools 1.17

However, upgrade the dotenv variable SAMTOOLS_VERSION to 1.18 and you'll see:

❯ nextflow -quiet run main.nf
samtools 1.18

Conveniently for debugging, local environment variables take precedence over dotenv file variables:

❯ SAMTOOLS_VERSION=1.16 nextflow -quiet run main.nf
samtools 1.16

Testing the Plugin Locally

Execute the following to compile and run unit tests for the plugin:

make test

To install the plugin for use in local workflows (e.g. not internet connected), execute the following:

make install 

Using the Plugin Locally

Test your changes to the plugin on a Nextflow script like:

nextflow/launch.sh run <script.nf> -plugins nf-dotenv@#.#.#-dev

Publishing to GitHub

Read and follow the official documentation on setting up your environment for plugin release.

After bumping the version of the plugin in the file build.gradle and making a GitHub Release, execute the following:

make release