Skip to content

Add Parabricks resource configuration guide for full-size genomes - #1775

Merged
pinin4fjords merged 4 commits into
devfrom
docs/parabricks-config-guide
May 11, 2026
Merged

Add Parabricks resource configuration guide for full-size genomes#1775
pinin4fjords merged 4 commits into
devfrom
docs/parabricks-config-guide

Conversation

@pinin4fjords

@pinin4fjords pinin4fjords commented Apr 8, 2026

Copy link
Copy Markdown
Member

Summary

Adds practical guidance for configuring Parabricks GPU resources when running full-size genomes, based on testing with the test_full dataset (GRCh37, 8 paired-end samples) on AWS g5 instances.

What's documented

  • Recommended Nextflow config for Parabricks processes (GPU count, memory scaling, retry strategy)
  • The --low-memory STAR flag and why it's needed for GPUs with 24GB VRAM
  • Why retries should request all GPUs on the node to prevent co-scheduling contention
  • Parabricks exit code 255 behavior (covers GPU OOM, host OOM, and internal errors)
  • Note about --genome incompatibility (references the fix in Fix iGenomes STAR version detection and index building logic #1756)

Framed as tested examples rather than prescriptive values, so users can adapt for their cloud provider and GPU type.

Context

Discovered through extensive benchmarking runs on Seqera Platform. Key failure modes encountered and resolved:

  • GPU OOM during stitchPiece_step0 without --low-memory on A10G GPUs
  • Host OOM (SIGKILL) when container memory limit was too low
  • GPU memory contention when Batch co-scheduled two 4-GPU tasks on an 8-GPU node
  • Parabricks segfault (signal 11) during CPU sorting phase (transient, resolved by retry)

Test plan

  • Docs render correctly on nf-co.re
  • Config example is valid Nextflow syntax

🤖 Generated with Claude Code

pinin4fjords and others added 2 commits April 8, 2026 13:54
@pinin4fjords @claude
docs: add Parabricks resource configuration guide for full-size genomes
e762356 
Adds practical guidance for configuring Parabricks GPU resources when
running full-size genomes, based on testing with the test_full dataset
(GRCh37, 8 samples) on AWS g5 instances. Covers GPU count, memory
scaling, retry strategy, --low-memory flag, and GPU contention
avoidance. Framed as tested examples rather than prescriptive values
so users can adapt for their cloud provider and GPU type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pinin4fjords @claude
docs: add changelog entry for Parabricks config guide
b22b3bc 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 8, 2026 •
edited
Loading

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 4c57932

+| ✅ 215 tests passed       |+
#| ❔  19 tests were ignored |#
!| ❗   7 tests had warnings |!
Details

❗ Test warnings:

  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • files_exist - File is ignored: conf/containers_conda_lock_files_amd64.config
  • files_exist - File is ignored: conf/containers_conda_lock_files_arm64.config
  • files_exist - File is ignored: conf/containers_docker_amd64.config
  • files_exist - File is ignored: conf/containers_docker_arm64.config
  • files_exist - File is ignored: conf/containers_singularity_https_amd64.config
  • files_exist - File is ignored: conf/containers_singularity_https_arm64.config
  • files_exist - File is ignored: conf/containers_singularity_oras_amd64.config
  • files_exist - File is ignored: conf/containers_singularity_oras_arm64.config
  • nextflow_config - Config default ignored: params.ribo_database_manifest
  • nf_test_content - nf_test_content
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: assets/nf-core-rnaseq_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-rnaseq_logo_dark.png
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore
  • actions_nf_test - actions_nf_test
  • modules_config - modules_config
  • container_configs - container_configs

✅ Tests passed:

Run details

  • nf-core/tools version 4.0.2
  • Run at 2026-05-11 08:31:03

@pinin4fjords pinin4fjords mentioned this pull request Apr 9, 2026
Merged
2 tasks
@pinin4fjords
Merge branch 'dev' into docs/parabricks-config-guide
4c57932
@pinin4fjords
docs(parabricks): scope withName to the alignment step [skip ci]
3596634 
The previous PARABRICKS_.* selector also matched PARABRICKS_STARGENOMEGENERATE
(a CPU-only STAR_GENOMEGENERATE alias, process_high), applying GPU resources
to the index build.
@pinin4fjords
pinin4fjords merged commit f4aa515 into dev May 11, 2026
2 checks passed
@pinin4fjords
pinin4fjords deleted the docs/parabricks-config-guide branch May 11, 2026 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@pinin4fjords