Dev -> master for nf-core/rnaseq 3.26.0#1838
Merged
Merged
Conversation
Swap trimgalore container from biocontainers/trim-galore:0.6.10 to ghcr.io/felixkrueger/trimgalore:dev (Rust rewrite, single binary). Remove --fastqc_args from TRIMGALORE preset args since the new container does not bundle FastQC (single-tool container philosophy). Raw-read FastQC still runs via the separate FASTQC process in the subworkflow.
The TrimGalore v2.0 dev container now includes FastQC 0.12.1, so the original --fastqc_args preset works. Only the container swap remains.
Replace version string 0.6.10 → 2.0.0 in all snapshot files. This is the only difference — trimming output is byte-identical.
Switches from the moving :dev tag to the immutable manifest-list digest for the v2.1.0-beta.1 release, so CI reproducibility is not a function of when the run starts. Multi-arch manifest resolves to linux/amd64 or linux/arm64 per runner automatically. See https://github.com/FelixKrueger/TrimGalore/releases/tag/v2.1.0-beta.1 Snapshots will need regeneration: v2.1.0-beta.1 reports its own version (was 2.0.0), and commit eedbc66 (post-2.0.0) reformatted the Cutadapt section of the text trimming report to match v0.6.x MultiQC-parsed values, so report-text MD5s will move. .fq.gz output MD5s should stay stable (byte-identical output is the rewrite guarantee).
Updates the module-level nf-test and snapshot for the Rust rewrite
(Oxidized Edition).
Two changes in tests/main.nf.test:
- Filter empty lines from log content via .findAll { it } before passing
to snapshot(). Works around an nft-utils v0.0.5 path-coercion bug where
empty String list elements resolve to CWD and trigger a recursive md5
dump of the entire working tree (produced a 132K-line corrupt snap).
- Snapshot now captures report text with report-format changes from
v2.1.0-beta.1: removed "Cutadapt version:" line, removed "Number of
cores used for trimming:", added Oxidized Edition banner, added the
hardcoded MultiQC-compat cutadapt identifier.
main.nf.test.snap regenerated with --update-snapshot.
Mechanical version replace across 18 pipeline-level nf-test snapshots: "trimgalore": "2.0.0" -> "trimgalore": "2.1.0" 44 substitutions in total. Cutadapt md5 drift is expected (MultiQC's cutadapt parser reads TrimGalore's trimming-report Cutadapt section, and commit eedbc66 in TrimGalore changed those numeric values to match v0.6.x MultiQC-parsed values) but not regen'd locally — STAR_ALIGN cannot run under Rosetta on Apple Silicon (FIFO filesystem incompatibility) so local pipeline regen is infeasible. CI will surface the real md5 drift across all 18 tests in parallel.
- Remove TRIMGALORE withName block from conf/arm.config that was pinning the legacy Cutadapt+TrimGalore 0.6.10 container, shadowing the multi-arch digest in modules/nf-core/trimgalore/main.nf - Update tests/gpu_ribodetector.nf.test.snap to expect trimgalore 2.1.0 (pulled in via the dev merge at the stored 0.6.10 value)
- .nf-core.yml: version 3.25.0 -> 3.26.0dev - nextflow.config: manifest.version 3.25.0 -> 3.26.0dev - ro-crate-metadata.json: url -> nf-co.re/rnaseq/dev/, version -> 3.26.0dev - workflows/rnaseq/assets/multiqc/multiqc_config.yml: flip report links back to dev; fix github.com/.../tag/<name> (404) to github.com/.../tree/<name> (200) which slipped into the 3.25.0 release - CHANGELOG.md: open a new 'dev - xxxx-xx-xx' section
Bump version after release 3.25.0
Merges upstream dev (128 files changed since branch) and resolves the three test-snap conflicts: - tests/min_mapped_reads.nf.test.snap — dev deleted (test removed upstream); take dev's deletion. - tests/skip_quantification_merge.nf.test.snap — same category; deleted. - tests/skip_qc.nf.test.snap — dev restructured the test cases (renamed from "--skip_qc - stub"/"--skip_qc" to "--skip_qc --min_mapped_reads 90"); take dev's version and bump the "trimgalore": "0.6.10" entries to "2.1.0" to match v2 integration. Also bumps "trimgalore": "0.6.10" → "2.1.0" in tests/salmon.nf.test.snap, which auto-merged cleanly but imported two new test-case blocks from dev that still referenced the Perl version. Container pin correctly at v2.1.0-beta.3 (sha256:3d8e9c90555cb47176ea4f7527230c59995f402af6a32d3ef7b3428ca5d6b337). The preceding commit 952027c was an empty no-op commit (PUT submitted content identical to what was already on the branch, so GitHub created a no-op commit); this merge commit is what actually applies the beta.3 digest.
Pulls in nf-core/modules#11308: - Defensive cleanup of stale trim_galore outputs at the top of both SE and PE script branches, so a same-workdir retry (AWS Batch retry after a Spot reclaim) cannot leave an orphan `*_trimmed.fq.gz` intermediate that breaks downstream FQ_LINT_AFTER_TRIMMING. - Pin cutadapt=5.2 in environment.yml to match the docker container. - Filter the python-version line out of the snapshotted log. Reported via #1807. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous \`rm -f *.fq.gz\` cleanup deleted Nextflow-staged input files
when those happened to use the \`.fq.gz\` extension (the prokaryotic test
profile inputs end in \`.fq.gz\`), leaving the module's symlinks pointing
at deleted targets and breaking trim_galore.
Replace with \`rm -f \${prefix}_1_trimmed.fq.gz \${prefix}_2_trimmed.fq.gz\`,
which targets exactly the orphan filenames trim_galore would have produced
and unlinked itself. Drop the cleanup from the SE branch entirely - SE
mode has no validate_paired_end_files step and therefore no intermediate
to leak.
This deviates from upstream nf-core/modules until a follow-up PR lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dc7e4ea (the beta.4 digest re-pin) accidentally dropped one level of backslash escaping throughout the file because the shell pipeline used to apply the digest substitution did backslash interpretation on the file content. This commit restores the file to byte-identical state with the pre-regression file aside from the (correct) beta.4 digest update. Concrete impact of the regression: - `args.split("\\s(?=--)")` had become `args.split("\s(?=--)")` — in Groovy that regex matches the literal letter "s" before `--` instead of whitespace. Single-end argument splitting was broken. - Several `trim_galore \\` line-continuations had dropped to `trim_galore \` — likely still functional but no longer byte-equivalent to the previous correct state. The digest pin to v2.1.0-beta.4 (sha256:9a7f7ddedde9df5400a1819fd22fd0fae20c8c9f41bbcd5ac7ea382178c794f0) is preserved.
nf-core/modules#11309 merged the same narrow-cleanup fix that's already applied in this PR. Bump the recorded git_sha so module_changes lint passes and the local trimgalore module matches upstream master. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…lore-fix Patch release 3.25.1: trimgalore same-workdir-retry cleanup
Dropped in #1823 on the rationale that the post-3.25.0 dev bump was being reverted by the 3.25.1 prep, so didn't belong in the 3.25.1 release notes. That was wrong - every PR that lands in dev should appear in the changelog. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tore-1821 docs(changelog): restore #1821 entry under 3.25.1
Per review feedback: as a user reading usage.md, the relevant question is "does it just work?", not how the upgrade adapter rewrites genomeParameters.txt internally. Drop the schema field names and process name from the iGenomes / custom-catalogue sections; keep the user-facing answer (yes for iGenomes, set star_legacy if mirroring 2.6.x content). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop legacy STAR 2.6.1d pin; upgrade iGenomes index params instead
Reinstalls trimgalore to pick up the upstream label change (process_high -> process_medium + process_low_memory) and adds the matching process_low_memory definition (1 GB) to conf/base.config from the nf-core/tools template. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reinstall trimgalore + add process_low_memory label to base.config
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bump version to 3.26.0 ahead of release
|
famosab
approved these changes
May 7, 2026
famosab
left a comment
famosab
left a comment
There was a problem hiding this comment.
I did not find any major things. The only thing I really was wondering about is the required nextflow version - I thought it was bumped by the template updtae to 4.0.2. Other things are more questions or nitpicks that can be addressed if wanted :)
Two actionable nitpicks from Famke's #1838 review: - Condense the two large `strandCheckSummaryYaml` JSON snapshots in `multiqc_rnaseq` function tests to `.md5()` so the snap file stops carrying ~1 KB of inlined JSON per test - Add a Software dependencies subsection to the 3.26.0 changelog so it is obvious at a glance which tool versions changed in this release (only `trim-galore`) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
Make it visually obvious in the table itself that the STAR removal covers only the parallel 2.6.1d legacy build, not the default aligner. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address review feedback from #1838
vagkaratzas
approved these changes
May 7, 2026
vagkaratzas
left a comment
vagkaratzas
left a comment
Contributor
There was a problem hiding this comment.
Awesome, nothing blocking! Just some comments here and there to have a look.
Doesn't nextflow lint * flag the Channel.of entries? There are a couple leftover
Three actionable points from Vag's #1838 review: - Lowercase Channel.x -> channel.x in local test files and workflows/rnaseq/main.nf. nf-core component changes deferred (would need an upstream PR per file) - pipeline_completion test: pin params.outdir = "${outputDir}" so Nextflow's report/timeline/trace/dag files (wired against params.outdir in nextflow.config) land in the nf-test sandbox instead of a literal null/pipeline_info/ directory. Verified on the VM - Populate the previously empty "Pipeline specific contribution guidelines" section in docs/CONTRIBUTING.md with rnaseq-specific notes: test profiles, CI skip env vars, test data sources, module conventions, .nftignore, snapshots, version-reporting topic, --genome catalogues, CHANGELOG style Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address review feedback from #1838 (Vag)
… ci] Upstream nf-core/modules relabelled TRIMGALORE from process_high (12 cpus) to process_medium + process_low_memory (6 cpus, 1 GB), and we picked that up via the reinstall in #1836. The module's internal --cores formula is task.cpus - 4 (paired) clamped to [1, 8], so the cpu downgrade also drops --cores from 8 to 2 and roughly halves trim_galore throughput on hosts that can spare the cores. Override cpus on the rnaseq-side selector to restore the previous behaviour without diverging from the upstream module. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
fix(trimgalore): restore --cores 8 by pinning TRIMGALORE to 12 cpus
Megatests on commit 979a7c2 (PR #1841, cpus=12 -> --cores 8) showed trim_galore only saturates ~2.78 cores on average (pcpu ~278%) per paired-end task, despite being given 12. The module's internal --cores formula (task.cpus - 4) was tuned for the original Perl/cutadapt/pigz wrapper where each --cores unit also implied a cutadapt subprocess and pigz I/O thread; with the Rust rewrite (trim_galore 2.x) those subprocesses are gone and the per --cores accounting overshoots the useful parallelism, so the empirical ceiling lands around 3 effective cores even at --cores 8. Drop the override to 8 cpus -> --cores 4 (paired). Wall-clock should be within a minute or two of --cores 8 since the effective core usage caps out around 3 either way, while freeing 4 cores per task for other work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix(trimgalore): drop TRIMGALORE override from 12 to 8 cpus
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 3.26.0
92 commits, 110 files changed since 3.25.0.
This release ships TrimGalore 2.1.0 (full Rust rewrite, byte-identical output, multi-arch including ARM), drops the parallel STAR 2.6.1d pin in favour of a metadata-only upgrade adapter for legacy iGenomes indices, syncs the nf-core template to v4.0.2, and lands several reproducibility and MultiQC fixes including augmented tx2gene publishing, lengthScaledTPM suppression under
--skip_quantification_merge, and FastQC General Stats column disambiguation.A short-lived 3.25.1 patch attempt was made then reverted, with its content folded back into 3.26.0; supersede relationships are noted inline below.
Review guide
Each component PR was individually reviewed before merging to dev.
New features
trimgaloremodule to 2.1.0 (nf-core/modules#11524) — full Rust rewrite, byte-identical trimming output, multi-arch container; add ARM Wave container for the new pin inconf/arm.config. Iteratively patched by #1823 (stale-workdir cleanup) and #1836 (process_low_memorylabel change)star_legacy = trueroute through a newSTAR_GENOMEPARAMS_UPGRADEprocess that rewrites the legacygenomeParameters.txtto the 2.7.4a schema, so stock STAR runs the modern build the pipeline ships; Sentieon and Parabricks STAR branches bypass the upgrade. RemovesSTAR_ALIGN_LEGACY, the parallel STAR 2.6.1d Wave/conda containers,conf/legacy_star.config, and the ARM-specific 2.6.1d override. Supersedes #1834: thestar_legacyflag plumbing that #1834 introduced is reused, but the 2.6.1d alignment alias #1834 was gating is now goneBug fixes
trimgaloreto pull in nf-core/modules#11308, which removes orphan*_trimmed.fq.gzoutputs left in the workdir by an interrupted previous trim_galore attempt (e.g. AWS Batch retry after a Spot reclaim) that were breakingFQ_LINT_AFTER_TRIMMING(#1807). Originally cut as the 3.25.1 patch release; that release bump was reverted in #1829 but the trimgalore reinstall content stayed in dev(raw),(trim)and(filt)(#1828). Also reverts the 3.25.1 release bump back to3.26.0dev*.merged.tx2gene_augmented.tsvalongside the GTF-derived*.merged.tx2gene.tsv; this is the table actually consumed bytximport(input mappings plus self-mappings for orphan transcripts) and is required to reproduce the published gene-level outputs from the per-sample quantification files (#1830)*gene_counts_length_scaled.tsvoutputs in--skip_quantification_mergemode (#1822)star_legacyflag so custom catalogues with modern indices skip it; rename the iGenomes-named scaffolding (STAR_ALIGN_IGENOMES,use_igenomes_star,conf/igenomes_star.config) tolegacy-prefixed equivalents; sort the merged Strandedness checks MultiQC tables by sample id for deterministic md5. Largely superseded by #1835, which removes the 2.6.1d pin entirely; thestar_legacyflag and rename remain in use, the strandedness sort fix standsRefactors and maintenance
trimgaloreto pick up upstream label change (process_high→process_medium+process_low_memory); add the matchingprocess_low_memorydefinition (1 GB) toconf/base.configper the nf-core/tools templateDocumentation and release prep
Test plan
🤖 Generated with Claude Code