Skip to content

Commit f4aa515

Browse files
authored
Merge pull request #1775 from nf-core/docs/parabricks-config-guide
Add Parabricks resource configuration guide for full-size genomes
2 parents c8a688b + 3596634 commit f4aa515

2 files changed

Lines changed: 37 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Enhancements and fixes
1111

12+
- [PR #1775](https://github.com/nf-core/rnaseq/pull/1775) - Add Parabricks resource configuration guide for full-size genomes (GPU count, memory scaling, retry strategy, `--low-memory` flag)
1213
- [PR #1844](https://github.com/nf-core/rnaseq/pull/1844) - Bump version to 3.27.0dev after release 3.26.0; flip the MultiQC report links and RO-Crate URL/version back to dev
1314
- [PR #1848](https://github.com/nf-core/rnaseq/pull/1848) - Align `actions/checkout` SHA in `nf-test-arm.yml` and `nf-test-gpu.yml` with the template-derived `nf-test.yml` (`v6`) ([#1847](https://github.com/nf-core/rnaseq/issues/1847))
1415
- [PR #1849](https://github.com/nf-core/rnaseq/pull/1849) - Tidy-up batch addressing [#1845](https://github.com/nf-core/rnaseq/issues/1845): README/schema clarifications, `&``&&` in three guards, deduplicated `withName` blocks, and removal of the unmaintained `bin/fastq_dir_to_samplesheet.py` (use [nf-core/fetchngs](https://nf-co.re/fetchngs) for samplesheet generation)

docs/usage.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -369,6 +369,42 @@ Parabricks `rna_fq2bam` is based on STAR 2.7.2a. The following native STAR flags
369369

370370
These differences are unlikely to materially affect downstream quantification results, but users should be aware of them for reproducibility purposes. All other STAR parameters (multi-mapping limits, intron sizes, mate gap, splice junction overhangs, etc.) have pbrun equivalents and are applied consistently.
371371

372+
#### Resource configuration for full-size genomes
373+
374+
The default resource configuration (`accelerator = 1` via the `process_gpu` label) is sized for small test datasets. Full-size genomes (e.g. GRCh37/GRCh38) require more GPUs, memory, and the `--low-memory` STAR flag to reduce GPU memory pressure.
375+
376+
The following config was tested with the `test_full` dataset (GRCh37, 8 paired-end samples) on AWS g5 instances (NVIDIA A10G, 24GB VRAM per GPU). Adjust for your cloud provider and GPU type:
377+
378+
```groovy
379+
process {
380+
withName: '.*ALIGN_STAR:PARABRICKS_RNA_FQ2BAM' {
381+
accelerator = { task.attempt > 1 ? 8 : 4 }
382+
cpus = { 48 * task.attempt }
383+
memory = { task.attempt > 1 ? 370.GB : 186.GB }
384+
resourceLimits = [cpus: 96, memory: 370.GB]
385+
maxRetries = 3
386+
errorStrategy = { task.exitStatus in [1,2,143,137,104,134,139,255] ? 'retry' : 'finish' }
387+
}
388+
}
389+
```
390+
391+
And pass the low-memory flag:
392+
393+
```bash
394+
--extra_star_align_args '--low-memory'
395+
```
396+
397+
Key considerations when adapting this for your environment:
398+
399+
- **Multiple GPUs are needed** for full-size genomes. 4 GPUs with `--low-memory` was sufficient for GRCh37 on A10G (24GB VRAM). GPUs with more VRAM (e.g. A100 40GB/80GB) may work with fewer GPUs or without `--low-memory`.
400+
- **Request all GPUs on retry** to prevent multiple Parabricks tasks being co-scheduled on the same node. GPU memory is not managed by the scheduler, so two 4-GPU tasks on an 8-GPU node will contend for VRAM and fail.
401+
- **Parabricks returns exit code 255 for most failures** including GPU OOM, host OOM, and internal errors. The retry strategy above handles this, but be aware that not all 255 exits are resource-related.
402+
- **Ensure your compute environment includes GPU instance types large enough for the retry**. For example, on AWS you might include both `g5.12xlarge` (4 GPUs, 192GB RAM) and `g5.48xlarge` (8 GPUs, 768GB RAM).
403+
404+
:::note
405+
Do not use `--genome` with `--use_parabricks_star`. The pre-built iGenomes STAR indices are incompatible with Parabricks' bundled STAR version. Supply `--fasta` and `--gtf` explicitly instead.
406+
:::
407+
372408
### RustQC: accelerated post-alignment QC (experimental)
373409

374410
:::warning

0 commit comments

Comments
 (0)