Merge pull request #398 from nf-core/dev

Release 4.3.1 update tutorials

Author: ggabernet

.github/workflows/awsfulltest.yml

@@ -4,61 +4,40 @@ name: nf-core AWS full size tests
 # It runs the -profile 'test_full' on AWS batch
 
 on:
-  pull_request:
-    branches:
-      - main
-      - master
   workflow_dispatch:
   pull_request_review:
     types: [submitted]
+  release:
+    types: [published]
 
 jobs:
   run-platform:
     name: Run AWS full tests
-    # run only if the PR is approved by at least 2 reviewers and against the master branch or manually triggered
-    if: github.repository == 'nf-core/airrflow' && github.event.review.state == 'approved' && github.event.pull_request.base.ref == 'master' || github.event_name == 'workflow_dispatch'
+    # run only if the PR is approved by at least 2 reviewers and against the master/main branch or manually triggered
+    if: github.repository == 'nf-core/airrflow' && github.event.review.state == 'approved' && (github.event.pull_request.base.ref == 'master' || github.event.pull_request.base.ref == 'main') || github.event_name == 'workflow_dispatch' || github.event_name == 'release'
     runs-on: ubuntu-latest
     steps:
-      - name: Get PR reviews
-        uses: octokit/request-action@v2.x
-        if: github.event_name != 'workflow_dispatch'
-        id: check_approvals
-        continue-on-error: true
-        with:
-          route: GET /repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews?per_page=100
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-
-      - name: Check for approvals
-        if: ${{ failure() && github.event_name != 'workflow_dispatch' }}
-        run: |
-          echo "No review approvals found. At least 2 approvals are required to run this action automatically."
-          exit 1
-
-      - name: Check for enough approvals (>=2)
-        id: test_variables
-        if: github.event_name != 'workflow_dispatch'
+      - name: Set revision variable
+        id: revision
         run: |
-          JSON_RESPONSE='${{ steps.check_approvals.outputs.data }}'
-          CURRENT_APPROVALS_COUNT=$(echo $JSON_RESPONSE | jq -c '[.[] | select(.state | contains("APPROVED")) ] | length')
-          test $CURRENT_APPROVALS_COUNT -ge 2 || exit 1 # At least 2 approvals are required
+          echo "revision=${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'release') && github.sha || 'dev' }}" >> "$GITHUB_OUTPUT"
 
       - name: Launch workflow via Seqera Platform
         uses: seqeralabs/action-tower-launch@v2
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
           access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
-          revision: ${{ github.sha }}
-          workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/airrflow/work-${{ github.sha }}
+          revision: ${{ steps.revision.outputs.revision }}
+          workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/airrflow/work-${{ steps.revision.outputs.revision }}
           parameters: |
             {
               "hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
-              "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/airrflow/results-${{ github.sha }}"
+              "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/airrflow/results-${{ steps.revision.outputs.revision }}"
             }
           profiles: test_full
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
         with:
           name: Seqera Platform debug log file
           path: |

CHANGELOG.md

@@ -3,6 +3,16 @@
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [4.3.1] - Revelio hotfix
+
+### `Added`
+
+- [#399](https://github.com/nf-core/airrflow/pull/399) Bump versions.
+
+### `Fixed`
+
+- [#392](https://github.com/nf-core/airrflow/pull/392) Updated tutorials.
+
 ## [4.3.0] - Revelio
 
 ### `Added`

docs/usage.md

@@ -6,7 +6,7 @@
 
 ## Introduction
 
-The nf-core/airrflow pipeline allows processing B-cell receptor (BCR) and and T-cell receptor (TCR) sequencing data from bulk and single-cell sequencing protocols. It allows the processing of targeted bulk and single-cell adaptive immune receptor sequencing data (AIRR-seq), as well as the extraction of TCR and BCR sequences from untargeted bulk and single-cell RNA-seq data. The pipeline enables and end-to-end analysis, departing from raw reads or readily assembled sequences, and performs sequence assembly, V(D)J assignment, clonal group inference, lineage reconstruction and repertoire analysis using the [Immcantation](https://immcantation.readthedocs.io/en/stable/) framework, as well as other immune repertoire analysis tools.
+nf-core/airrflow allows processing B-cell receptor (BCR) and and T-cell receptor (TCR) sequencing data from bulk and single-cell sequencing protocols. It allows the processing of targeted bulk and single-cell adaptive immune receptor sequencing data (AIRR-seq), as well as the extraction of TCR and BCR sequences from untargeted bulk and single-cell RNA-seq data. The pipeline enables and end-to-end analysis, departing from raw reads or readily assembled sequences, and performs sequence assembly, V(D)J assignment, clonal group inference, lineage reconstruction and repertoire analysis using the [Immcantation](https://immcantation.readthedocs.io/en/stable/) framework, as well as other immune repertoire analysis tools.
 
 In addition to this page, you can find additional information on how to use the pipeline on the following pages:
 
@@ -198,10 +198,10 @@ An example samplesheet is:
 
 It is possible to provide several fastq files per sample (e.g. sequenced over different chips or lanes). In this case the different fastq files per sample will be provided to the same cellranger process. These rows should then have an identical `sample_id` field.
 
-### Fastq input samplesheet (untargeted bulk or single-cell RNAseq)
+### Fastq input samplesheet (untargeted bulk or single-cell RNA-seq)
 
 When running the untargeted protocol, BCR or TCR sequences will be extracted from the untargeted bulk or single-cell RNA sequencing with tools such as [TRUST4](https://github.com/liulab-dfci/TRUST4).
-The required input file is the same as for the [Fastq bulk AIRR samplesheet](#fastq-input-samplesheet-bulk-airr-sequencing) or [Fastq single-cell AIRR samplesheet](#fastq-input-samplesheet-single-cell-sequencing) depending on the input data type (bulk RNAseq or single-cell RNAseq).
+The required input file is the same as for the [Fastq bulk AIRR samplesheet](#fastq-input-samplesheet-bulk-airr-sequencing) or [Fastq single-cell AIRR samplesheet](#fastq-input-samplesheet-single-cell-sequencing) depending on the input data type (bulk RNA-seq or single-cell RNA-seq).
 
 ### Assembled input samplesheet (bulk or single-cell sequencing)
 
@@ -535,7 +535,7 @@ nextflow run nf-core/airrfow \
 ```
 
 - If UMI's are present, the read containing them must be specified using the `--umi_read` parameter.
-- The `--read_format` parameter can be used to specify the Cell Barcode and UMI position within the reads (see TRUST4 [docs](https://github.com/liulab-dfci/TRUST4?tab=readme-ov-file#10x-genomics-data-and-barcode-based-single-cell-data)). For scRNAseq with 10X Genomics the R1 read usually contains both the cell barcode (barcode) and UMI. So we specify "R1" for both `--umi_read` and `--cell_barcode_read`, and the positions of both the cell barcode and UMI with the `--read_format` parameter as in the example ("bc:0:15,um:16:27"). Then specify the R1 read in the filename_R1 column of the samplesheet, and the read containing the actual sequence (usually R2) in the filename_R2 column of the samplesheet.
+- The `--read_format` parameter can be used to specify the Cell Barcode and UMI position within the reads (see TRUST4 [docs](https://github.com/liulab-dfci/TRUST4?tab=readme-ov-file#10x-genomics-data-and-barcode-based-single-cell-data)). For scRNA-seq with 10X Genomics the R1 read usually contains both the cell barcode (barcode) and UMI. So we specify "R1" for both `--umi_read` and `--cell_barcode_read`, and the positions of both the cell barcode and UMI with the `--read_format` parameter as in the example ("bc:0:15,um:16:27"). Then specify the R1 read in the filename_R1 column of the samplesheet, and the read containing the actual sequence (usually R2) in the filename_R2 column of the samplesheet.
 
 ## Important considerations for clonal analysis
 
@@ -622,7 +622,7 @@ Specify the path to a specific config file (this is a core Nextflow command). Se
 
 ### Resource requests
 
-Whilst the default requirements set within the pipeline will hopefully work for most people and with most input data, you may find that you want to customise the compute resources that the pipeline requests. Each step in the pipeline has a default set of requirements for number of CPUs, memory and time. For most of the pipeline steps, if the job exits with any of the error codes specified [here](https://github.com/nf-core/rnaseq/blob/4c27ef5610c87db00c3c5a3eed10b1d161abf575/conf/base.config#L18) it will automatically be resubmitted with higher resources request (2 x original, then 3 x original). If it still fails after the third attempt then the pipeline execution is stopped.
+Whilst the default requirements set within the pipeline will hopefully work for most people and with most input data, you may find that you want to customise the compute resources that the pipeline requests. Each step in the pipeline has a default set of requirements for number of CPUs, memory and time. For most of the pipeline steps, if the job exits with any of the error codes specified [here](https://github.com/nf-core/airrflow/blob/132ab3d129c0df3f2de0ede7a7afaf549277c512/conf/base.config#L17) it will automatically be resubmitted with higher resources request (2 x original, then 3 x original). If it still fails after the third attempt then the pipeline execution is stopped.
 
 To change the resource requests, please see the [max resources](https://nf-co.re/docs/usage/configuration#max-resources) and [tuning workflow resources](https://nf-co.re/docs/usage/configuration#tuning-workflow-resources) section of the nf-core website.
 

docs/usage/FAQ.md

@@ -0,0 +1,52 @@
+# nf-core/airrflow: Frequently Asked Questions
+
+## How to update process resource requests and resource limits?
+
+By default, the pipeline defines reasonable resource requests for each process (number of CPUs, RAM memory, time limits) based on typical compute environments. However, you can adjust these settings to better match the size of your datasets or the capabilities of your compute infrastructure. You can customize the limits and requests in `resource.config` file and provide it to the pipeline using the -c parameter during execution. The `resourceLimits` option applies upper resource request limits to all the processes in the pipeline. Ensure that these limits do not exceed the available resources on your compute system.
+
+```json title="resource.config"
+process {
+   resourceLimits = [cpus: 8, memory: 72.GB, time: 24.h]
+}
+```
+
+To update the resource requests for a specific pipeline process, you can also provide specific process requests in this config file. For example, to update the resource requests for the `CHANGEO_ASSIGNGENES` process:
+
+```json title="resource.config"
+process {
+   resourceLimits = [cpus: 8, memory: 72.GB, time: 24.h]
+
+   withName:CHANGEO_ASSIGNGENES {
+        cpus   = 2
+        memory = 10.GB
+        time   = 5h
+   }
+}
+```
+
+In nf-core pipelines, each process has a label indicating the resources that are being requested (`process_low`, `process_medium`, `process_high`, ...). The CPUs, RAM and time set up for each of these labels can be found in the [base.config](https://github.com/nf-core/airrflow/blob/master/conf/base.config) file. You can update the resource requests for all processes with a specific label by providing the updated configuration. For example here we update the resource requests of processes with the `process_high` label:
+
+```json title="resource.config"
+process {
+   resourceLimits = [cpus: 24, memory: 100.GB, time: 24.h]
+
+   withLabel:process_high {
+        cpus   = 24
+        memory = 100.GB
+        time   = 10h
+   }
+}
+```
+
+Note that the resource requests will never exceed what is specified in the `resourceLimits` line, so if you do want to increase the resource requests for specific processes, you should also increase the `resourceLimits` requests and run the pipeline in a compute infrastructure with sufficient resources. In this example we also have updated the `resourceLimits` to reflect that.
+
+> [!TIP]
+> For more information about nf-core pipeline resource configurations, check out the [nf-core pipeline configuration docs](https://nf-co.re/docs/usage/getting_started/configuration).
+
+## How to customize the analysis and figures?
+
+nf-core/airrflow is a standardized pipeline that performs the different computational analysis steps and provides standard figures for a first data exploration. You can use nf-core/airrflow results as input for customized analyses using R and the Immcantation tools. There are three options to customize your analysis:
+
+- Option 1: some of the intermediate analysis steps are stored on `RData` objects that can be loaded in R to customize your figures. For instance, clonal abundance calculations can be time-consuming, so the results are stored in the results folder (`clonal_abundance/define_clones/all_reps_clone_report/ggplots/abundanceSample.RData`). With `load()` function in R, both the abundance plot and the clonal abundance object can be loaded.
+- Option 2: perform your own downstream analysis with the Immcantation framework. You can load the nf-core/airrflow results in AIRR format in R and use the Immcantation tools to plot the data as you need for publications. Check the [Immcantation tutorials](https://immcantation.readthedocs.io/en/stable/getting_started/getting-started.html) for this purpose, e.g. the Immcantation's single-cell V(D)J analysis [here](https://immcantation.readthedocs.io/en/stable/getting_started/10x_tutorial.html) shows an example for single-cell data analysis.
+- Option 3: for more advanced users and in case you need to repeat the exact same analysis for multiple projects, you can customize the [Airrflow report](https://github.com/nf-core/airrflow/blob/master/assets/repertoire_comparison.Rmd) Rmarkdown file that comes with the pipeline and provide the updated version to the pipeline with the `--report_rmd` option. The pipeline will then use this file instead to create the report.