Subworkflow update

.nf-core.yml

@@ -48,4 +48,4 @@ template:
     - seqera_platform
     - multiqc
     - rocrate
-  version: 1.5.1
+  version: 1.6.0

.pre-commit-config.yaml

@@ -14,6 +14,7 @@ repos:
           (?x)^(
               .*ro-crate-metadata.json$|
               modules/nf-core/.*|
+              modules/sanger-tol/.*|
               subworkflows/nf-core/.*|
               .*\.snap$
           )$
@@ -22,6 +23,7 @@ repos:
           (?x)^(
               .*ro-crate-metadata.json$|
               modules/nf-core/.*|
+              modules/sanger-tol/.*|
               subworkflows/nf-core/.*|
               .*\.snap$
           )$

CHANGELOG.md

@@ -3,7 +3,67 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [[1.5.1]](https://github.com/sanger-tol/curationpretext/releases/tag/1.5.1)] - UNSC Punic (H1) - [2025-10-01]
+## [[1.6.0](https://github.com/sanger-tol/curationpretext/releases/tag/1.6.0)] - UNSC Trafalgar - [2025-02-19]
+
+## Added and Fixed
+
+- Template update to 3.5.2.
+- - The previous `GENERATE_MAPS` subworkflow has been replaced with `ALIGN_CRAM` and `CREATE_MAPS_{STDRD,HIRES}` (renamed from `CRAM_MAP_ILLUMINA_HIC` and `PAIRS_CREATE_CONTACT_MAPS`, from the [`sanger-tol/nf-core-modules`](https://github.com/sanger-tol/nf-core-modules) repository, respectively)
+- Files can now be given explicitly in the `--reads` parameter in the format of `[<file1>, <file2>, ...]`, alternatively it can accept a FOFN (File of file names).
+- Files can now be given explicitly in the `--cram` parameter in the format of `[<file1>, <file2>, ...]`, alternatively it can accept a FOFN (File of file names).
+- `--pre_mapped_bam` parameter added in order to supply 1 pre-mapped BAM file, in this case `--cram` would be empty.
+  - Warnings have been added to ensure:
+    - Only 1 pre-mapped BAM file is provided if `--pre_mapped_bam` is used.
+    - Only 1 of `--pre_mapped_bam` or `--cram` is used`
+- `--cram_chunk_size` parameter added by `ALIGN_CRAM` to make cram chunking configurable, defaulting to 10000.
+- `LONGREAD_COVERAGE` subworkflow has been updated to accept an array list of files.
+- Major Update to modules coinciding with changes to use Nextflow topics
+- Update to move all modules/subworkflows to version topics.
+  - Required a small change to the template topic collection otherwise it would fail as there is no ch_versions channel.
+- Update docs to include the features from the past few releases.
+- Remove duplicated `selected_aligner` code from `PIPELINE_INITIALISATION`.
+- Change install for `TELOMERE` modules so that we use the `SANGER-TOL` repository rather than local.
+- Removed now unused `bin` files.
+- Removed and replaced pipeline graph with new version.
+
+### Parameters
+
+| Old Version | New Versions      |
+| ----------- | ----------------- |
+| NA          | --pre_mapped      |
+| NA          | --cram_chunk_size |
+
+### Software Dependencies
+
+Note, since the pipeline is using Nextflow DSL2, each process will be run with its own Biocontainer. This means that on occasion it is entirely possible for the pipeline to be using different versions of the same tool. However, the overall software dependency changes compared to the last release have been listed below for reference.
+
+| Module                         | Old Version      | New Versions                                                   |
+| ------------------------------ | ---------------- | -------------------------------------------------------------- |
+| `BEDTOOLS_BAMTOBED`            | 2.30.0           | 2.31.1                                                         |
+| `BEDTOOLS_GENOMECOV`           | 2.30.0           | 2.31.1                                                         |
+| `BEDTOOLS_INTERSECT`           | 2.30.0           | 2.31.1                                                         |
+| `BEDTOOLS_MAKEWINDOWS`         | 2.30.0           | 2.31.1                                                         |
+| `BEDTOOLS_MAP`                 | 2.30.0           | 2.31.1                                                         |
+| `CRAMALIGN_BWAMEM2ALIGNHIC`    | NEW_ADDITION     | bwamem2: 2.2.1, samtools: 1.22.1                               |
+| `GAWK`                         | 5.2.0            | 5.3.0                                                          |
+| `GNU_SORT`                     | 9.1              | 9.5                                                            |
+| `MINIMAP2_ALIGN`               | 2.28--he4a0461_0 | 2.29-r1283                                                     |
+| `PRETEXTMAP`                   | 0.1.9            | 0.1.9 (Temporary Patch, to be updated to 0.2.4 once available) |
+| `SAMTOOLS_FAIDX`               | 1.21.2           | 1.22.1                                                         |
+| `SAMTOOLS_MERGE`               | 1.21.2           | 1.22.1                                                         |
+| `SAMTOOLS_SORT`                | 1.21.2           | 1.22.1                                                         |
+| `SAMTOOLS_SPLITHEADER`         | 1.21.2           | 1.22.1                                                         |
+| `SAMTOOLS_VIEW_FILTER_PRIMARY` | 1.21.2           | 1.22.1                                                         |
+| `SAMTOOLS_MERGEDUP`            | NEW_ADDITION     | 1.23.0                                                         |
+| `FIND_TELOMERE_WINDOWS`        | 1.0.0            | REMOVED                                                        |
+| `TELOMERE_WINDOWS`             | NEW_ADDITION     | 1.0.0                                                          |
+| `FIND_TELOMERE_REGIONS`        | 1.0.0            | REMOVED                                                        |
+| `TELOMERE_REGIONS`             | NEW_ADDITION     | 1.0.0                                                          |
+| `EXTRACT_TELOMERE`             | 1.0.0            | REMOVED                                                        |
+| `TELOMERE_EXTRACT`             | NEW_ADDITION     | 1.0.0                                                          |
+| `UCSC_BEDGRAPHTOBIGWIG`        | 447              | 469                                                            |
+
+## [[1.5.1](https://github.com/sanger-tol/curationpretext/releases/tag/1.5.1)] - UNSC Punic (H1) - [2025-10-01]
 
 ### Added and Fixed
 
@@ -114,8 +174,10 @@ Note, since the pipeline is using Nextflow DSL2, each process will be run with i
 - Updated all modules, versions which are the same indicate that the nf-core modules `.nf` has been updated without updating the tool.
 - Update modules and base config files for parity with TreeVal (large genome optimisations).
 - Update the PretextGraph version.
+- Change how params are used in the pipeline, now passed down from main workflow rather than used when ever needed.
+- Removed unnecessary schema file.
 
-### Paramters
+### Parameters
 
 | Old Version | New Versions  |
 | ----------- | ------------- |
@@ -135,8 +197,8 @@ Note, since the pipeline is using Nextflow DSL2, each process will be run with i
 | BAMTOBEDSORT        | 2.31.1 + 1.17      | REMOVED                              |
 | TABIX_BGZIPTABIX    | 1.20--h5efdd21_2   | REMOVED                              |
 | BWAMEM2_INDEX       | 2.2.1              | 2.2.1 (samtools=1.2.1, htslib=1.2.1) |
-| SAMTOOLS_FAIDX      | 1.2.1              | 1.2.1                                |
-| SAMTOOLS_VIEW       | 1.2.1              | 1.2.1                                |
+| SAMTOOLS_FAIDX      | 1.22.1             | 1.22.1                               |
+| SAMTOOLS_VIEW       | 1.22.1             | 1.22.1                               |
 | PRETEXT_GRAPH       | 0.0.8              | 0.0.9                                |
 
 ## [[1.3.2](https://github.com/sanger-tol/curationpretext/releases/tag/1.3.2)] - UNSC Pillar-of-Autumn (H2) - [2025-04-05]

README.md

@@ -17,11 +17,15 @@
 
 This is intended as a supplementary pipeline for the [treeval](https://github.com/sanger-tol/treeval) project. This pipeline can be simply used to generate pretext maps, information on how to run this pipeline can be found in the [usage documentation](https://pipelines.tol.sanger.ac.uk/curationpretext/usage).
 
-![Workflow Diagram](./docs/images/CurationPretext_1_3_0.png)
+![Workflow Diagram](./docs/images/CurationPretext-1.6.0.jpeg)
 
-1. Generate Maps - Generates pretext maps as well as a static image.
+The above image shows the use of this pipeline inside of the manual curation process and follows the below major steps.
 
-2. Accessory files - Generates the repeat density, gap, telomere, and coverage tracks.
+1. CRAM_MAP_ILLUMINA_HIC (ALIGN_CRAM) + PAIRS_CREATE_CONTACT_MAPS (CREATE_MAPS) - Generates pretext maps as well as a static image.
+
+2. ACCESSORY_FILES - Generates the repeat density, gap, telomere, and coverage tracks.
+
+3. PRETEXT_INGEST - Imports the generated tracks into pretext for visualisation.
 
 ## Usage
 
@@ -44,7 +48,7 @@ Currently, the pipeline uses the following flags:
   - The type of longread data you are utilising, e.g., ont, illumina, hifi.
 
 - `--aligner`
-  - The aligner yopu wish to use for the coverage generation, defaults to bwamem2 but minimap2 is also supported.
+  - The aligner you wish to use for the coverage generation, defaults to `AUTO` but options include `bwamem2` and `minimap2`.
 
 - `--cram`
   - The directory of the cram _and_ cram.crai files, e.g., `/path/to/cram/`
@@ -61,6 +65,18 @@ Currently, the pipeline uses the following flags:
 - `--all_output`
   - An option to output all maps + accessory files, the default will only output the pretextmaps where ingestion has occured.
 
+- `--skip_tracks`
+  - A csv list of accessory tracks to skip, options are: `ALL`, `gap`, `coverage`, `telo`, `repeats`, `NONE`. Default is `NONE`. Please note that capitalization matters.
+
+- `--split_telomere`
+  - A boolean to also generate the telomere track in 5Prime and 3Prime styles, this is also include the original telomere track.
+
+- `--pre_mapped_bam`
+  - A boolean option to use `--cram` as input for _A_ pre-mapped bam file.
+
+- `--cram_chunk_size`
+  - The number of records in a cram file should be chunked into, defaults to 10000.
+
 Now, you can run the pipeline using:
 
 ```bash
@@ -72,7 +88,7 @@ nextflow run sanger-tol/curationpretext \
   --sample { default is "pretext_rerun" } \
   --teloseq { default is "TTAGGG" } \
   --map_order { default is "unsorted" } \
-  --multi_mapping { default is "0" (for no mapping)} \
+  --multi_mapping { default is "0" (for no filtering of multi-mapping reads)} \
   --all_output <true/false> \
   --outdir { OUTDIR } \
   -profile <docker/singularity/{institute}>

bin/extract_repeat.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+
+"""
+A rewrite of the extract_repeat.pl (PERL) script.
+Original script written by: Yumi Sims (yy5)
+Rewritten by: Damon-Lee Pointon (dp24)
+
+Move through repeats file, line by line, and extract repeat information.
+"""
+
+import re
+import sys
+
+def main() -> None:
+    if len(sys.argv) < 2:
+        sys.exit("Usage: extract_repeat.py <file>")
+
+    file_path = sys.argv[1]
+    last = None
+
+    with open(file_path, "r", encoding="utf-8") as fh:
+        for line in fh:
+            line = line.rstrip("\n")
+            matched = re.match(r">(\S+)", line)
+            if matched:
+                last = matched.group(1)
+                continue
+
+            matched = re.match(r"(\d+)\s+-\s+(\d+)", line)
+            if matched:
+                print(f"{last}\t{matched.group(1)}\t{matched.group(2)}")
+                continue
+
+            sys.exit(f"Error --> {line}")
+
+if __name__ == "__main__":
+    main()