Skip to content

Docker: deprecate docker.userEmulation and use docker.runOptions; mitigates Trim Galore pipe freeze #435

@yxkwgp

Description

@yxkwgp

Description of the bug

Summary

When running the pipeline with -profile docker, two related issues occur:

  1. Deprecated config: Nextflow logs a warning that docker.userEmulation is no longer supported and should be removed from the config.
  2. Trim Galore can freeze: The Trim Galore step sometimes hangs in the cutadapt stage when run inside Docker. This is a known upstream issue (Trim Galore #207) caused by small pipe buffers in container environments.

Root cause (Trim Galore freeze)

  • By default, Docker containers run as root (UID 0). The Linux kernel accounts pipe buffer usage per UID.
  • When the total pipe usage for root exceeds pipe-user-pages-soft, new pipes are limited to 2 pages (8192 bytes) instead of the default 16 pages (~64 KB).
  • With many Nextflow tasks (each creating pipes, e.g. cutadapt ↔ trim_galore), root’s quota is easily exceeded, so new pipes get the small size and Trim Galore can block waiting for the last few lines from cutadapt (see Trim Galore #207).
  • Running containers as the host user (non-root) spreads pipe usage by UID and often avoids hitting that limit, which can mitigate or avoid the freeze.

Current config (problematic)

In nextflow.config, the docker profile uses the deprecated option:

docker {
docker.enabled = true
docker.userEmulation = true // deprecated, no longer supported by Nextflow
...
}

Suggested change

Remove docker.userEmulation = true from the docker profile (to clear the warning and stop relying on removed behaviour).
Add docker.runOptions = '-u $(id -u):$(id -g)' to the docker profile so containers run as the current user.

Command used and terminal output

nextflow -bg -log test.log -q run ~/pipeline/nf-core/atacseq-2.1.2 -profile docker -params-file params.json -resume

Relevant files

No response

System information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions