New pipeline: nf-core/hapmal

[igiraclement]

Pipeline title/name

hapmal

Keywords

Malaria, genomics, haplotype calling, Nextflow

What is it about?

It unifies processing of both short-read (Illumina) and long-read (Oxford Nanopore Technologies, ONT) data in a single, reproducible, containerized workflow for compositional profiling in multiclonal P. falciparum samples

Please provide a schematic diagram of the proposed pipeline

What would a minimal first release of this pipeline include?

Quality control of raw FASTQ files (FastQC)
Host read removal (BBDuk)
Alignment to the P. falciparum reference genome (Minimap2, works for both short and long reads)
Variant calling optimized for Plasmodium (Clair3 for long reads and basic Snippy support for short reads)
Basic haplotype profiling and compositional analysis of multiclonal infections (PHARE-inspired module)
Generation of a simple summary report (Nextflow and MultiQC-style output)

I confirm my proposed pipeline will follow nf-core guidelines. Most importantly, my pipeline will:

• be built with Nextflow.
• pass nf-core lint tests and use standardized parameters.
• be community-owned and developed within the nf-core organization.
• open source under the MIT license with proper credits and acknowledgments.
• have a descriptive, all lowercase, and without punctuation name.
• use the nf-core pipeline template and predominantly use official nf-core modules.
• focus on a specific data/analysis type with appropriate scope.
• have properly maintained documentation.
• be bundled using versioned Docker/Singularity containers.

Why do we need a new pipeline?

Existing nf-core pipelines such as pathogensurveillance provide excellent general frameworks for pathogen identification and variant detection. However, they do not specifically address the unique biological and technical challenges of malaria genomic surveillance.
HapMal will fill a critical gap by delivering the first nf-core pipeline dedicated to Plasmodium species (primarily P. falciparum), with a strong focus on hybrid short-read (Illumina) and long-read (ONT) data in a single unified workflow for for compositional profiling in multiclonal P. falciparum samples

Who would be interested?

Malaria genomic surveillance researchers and laboratories in malaria-endemic countries, Public health agencies and national malaria control programmes, Bioinformaticians and computational biologists working on Malaria and Field-based or low-resource setting teams who need portable, offline-capable, containerized workflows

What has been done so far

We have not yet started the development of the HapMal pipeline code.
The project is currently at the concept and planning stage. We have carefully reviewed the existing literature on malaria genomic surveillance, identified key gaps in hybrid short- and long-read analysis, and designed the overall architecture of the pipeline with a strong emphasis on compositional profiling of multiclonal P. falciparum infections.
We have assembled a small core team consisting of researchers and bioinformaticians with experience in basic to intermediate Nextflow skills. The team includes Nextflow ambassadors who are familiar with nf-core best practices, containerization (Docker/Singularity), and workflow development. This gives us a solid foundation to follow nf-core standards from the very beginning.

URL to existing work (if applicable)

No response

Are there any similar existing nf-core pipelines?

No response

[nf-core-bot]

Approval status: 🕐 Pending

Required approvals: Either 2 core team members OR 1 core team member + 1 maintainer

Review Status	Team members
🕐 Pending (Core)	@ewels, @maxulysse, @mribeirodantas, @mashehu, @JoseEspinosa, @mirpedrol, @matthdsm, @FriederikeHanssen, @kenibrewer, @jfy133, @atrigila, @edmundmiller, @awgymer, @SPPearce, @sateeshperi, @FranBonath, @LouisLeNezet, @christopher-hakkaart, @nvnieuwk
🕐 Pending (Maintainer)	@pontus, @maxulysse, @lpantano, @pinin4fjords, @mashehu, @JoseEspinosa, @mirpedrol, @GallVp, @FloWuenne, @matthdsm, @FriederikeHanssen, @adamrtalbot, @mahesh-panchal, @jfy133, @atrigila, @prototaxites, @ramprasadn, @edmundmiller, @luisas, @SPPearce, @camlloyd, @vagkaratzas, @sateeshperi, @jonasscheid, @famosab, @nictru, @itrujnara, @LouisLeNezet, @Joon-Klaps, @jasmezz, @Felix-Kummer, @sofstam, @nvnieuwk

[jfy133]

thanks for the proposal @igiraclement !

At a first glance, I can definitely see the distinction from pathogensurveillance, which is assembly focused rather than alignment based.. but then the different with e.g. bactmap is much lower: https://nf-co.re/bactmap/1.0.0/ (also does alignment, variant calling, tree building). Conceptually you could just add additional tools to the existing pipeline (although it looks like this is technically abandoned now... resurrecting would be nice!).

However I'm struggling to see what is exactly specific to P. falciparum versus other similar pathogens? Maybe a more detailed diagram would help - e.g. you mention Claire3 and PHARE but neither are in the diagram.
Also what does the 'resistance' part do exactly?

To me, you could have a more general pipeline which just switches out the reference genome and maybe a guidance tree for the particular pathogen of interest.

(Note if we do go with keeping this proposal as is, I would suggest calling it something with a similar prefix to the upcomign nf-core/plasmodiumdrugres pipeline, e.g. plasmodiumhaplotree or something like that)

[jfy133]

Bump on this @igiraclement , any thoughts on the above?

[igiraclement]

Hello @jfy133 . We are working on it. Hope to submit the proposal with all the comments addressed this coming week

[igiraclement]

Dear James, thanks for the bump. Just to clarify, is there a specific link or platform through which we should submit or reply?

We have already created a document incorporating both your comments and our responses. Could you please let us know the preferred modality for replying or sharing it back with you?

[mashehu]

just add your comments here and maybe update the initial proposal if necessary

[igiraclement]

Thanks @mashehu

[igiraclement]

nf core plasmodiumhapres proposal.docx

Dear nf-core team and community,
Following the previous comments and feedback on the hapmal proposal, we have now addressed them.
I have uploaded:

1. The pipeline MetroMap image (workflow diagram)
2. A document containing detailed answers to the comments and suggestions raised

Both files are attached to this comment.
I would appreciate it if you could review them and provide any further feedback. This pipeline aims to support malaria genomic surveillance, particularly for Plasmodium species drug resistance monitoring and haplotype analysis.
Thank you very much for your time and support!
Best regards,

[mashehu]

@igiraclement answer copy-pasted from the word doc, to save clicks:

Thank you for your feedback, James. To start, we have replaced our previous diagram with a metromap-style one that hopefully clears up what each part is doing.

Our proposed pipeline (renamed plasmodiumhapres) differs from nf-core/bactmap in several aspects which we list below. To summarise, we propose a tailored malaria genomic surveillance framework specifically designed for hybrid sequencing, field deployment, and public health integration, while bactmap is primarily a general bacterial read mapping and variant calling workflow.

Key Differences Between Our Proposed Pipeline and nf-core/bactmap

Our proposed pipeline introduces several innovations not commonly unified in existing workflows:

1. Plasmodiumhapres will deal with host reads

Our proposed pipeline will need two exchangeable reference files rather than one. Malarial Plasmodium is carried and transmitted by different Anopheles (mosquito) species, whose reads should be removed from the input .fastq files using a reference .fasta file. Different regions have species of both Plasmodium and Anopheles which are most relevant to their regional malaria transmission. We therefore propose to develop a pipeline which gives users flexibility and control over both pathogen and host reference genomes.

2. Hybrid Sequencing Integration

Most malaria pipelines process either:

-Illumina short reads, OR

-Nanopore long reads

Our pipeline aims to not only process both read types but also integrate both within a unified Nextflow workflow when hybrid analysis is possible.

This is important because:

- Long read sequencing can resolve full-length genes, useful for resolving haplotypes of repetitive AT-rich regions of malarial Plasmodium genomes

- Short read sequencing provides more accurate resolution of homopolymeric tracts, which are especially common in Plasmodium, including at antimalarial drug resistance genes

- Combined analysis improves structural resolution and variant detection

3. Malaria-Specific Genomic Surveillance

Unlike bacterial pipelines, our framework is tailored for:

Malaria drugs resistance monitoring:

• hrp2/hrp3 deletion analysis
• pfk13 resistance monitoring, pfcrt/pfmdr1 surveillance
• Vaccine antigen diversity
• Complexity of infection estimation
• Transmission genomics

Which are malaria-endemic-country priorities.

4. Designed for Low-Resource Settings

Field deployment in endemic settings is a major purpose of our proposed pipeline. We will therefore treat resource optimisation of our pipeline as critical rather than as a “nice-to-have”

5. Integrated Surveillance Ecosystem

Our proposal goes beyond variant calling.

The framework also aims to generate:

• Structured reports
• Epidemiological integration

Diagram shows step descriptions of our proposed pipeline including key intention of our work to pass results back through real world malaria surveillance settings. This will allow us to maintain and update how the report is generated in each version, preserving and improving relevance and completeness.

This shifts the pipeline from “bioinformatics software” toward a surveillance platform.

6. Multiclonal Infection Analysis

Malaria infections are often polyclonal.

Most generic mapping pipelines do not:

• Deconvolute mixed infections
• Estimate haplotype compositions
• Stratify patients by haplotype compositions

PHARE-inspired modules specifically target this challenge (Multiclonal Infection Profiling module), as will constant validation of this pipeline on regional publics mentioned in (4).

Also mentioned in (4), polyclonality of malaria infections and the necessity of their tracking in pathogen surveillance will also require bespoke handling in our proposed pipeline’s report compilation step.

Sustainability

This team consists of 4 nextflow affiliates, one member finishing a bioinformatics PhD studying a noncommunicable disease, one employed at Wellcome Sanger, one bachelors student and one fresh bachelors graduate, as well as 2 self-employed bioinformatics professionals. We are already applying for institutional affiliation and have ongoing focus on identifying and implementing other safeguards so that the project is not stalled or abandoned.

[jfy133]

Hi @igiraclement

Thanks for the extensive response, and in particular the updated metromap style diagram - I am a little ocnfused though, the metromap seems to drop a lot of tools ( e.g. Medeka), is that expected?

Regardless, if I base the evaluation on the metromap, this still does not seem to be Malaria specific other than the inclusion of PHARE (which looks to be a snakemake pipeline, is that correct?)

You could still have this pipeline organisim generic and have a user-selected option for picking PHARE + a custom report module .

And with that logic, I still feel there is still too much overlap with bactmap to warrant a separate pipeline - it would make more sense to me (when following nf-core principles) to support bactmap by adding and extending by adding tool options and extending it with optional end points (e.g. malaria specific bits).

With such a large team (Which is great already!) it would realy help to give the existing bactmap pipeline some TLC (e.g. updating the code base, adding support for modern long read data etc.).

@nf-core/maintainers any thoughts from you?

[BilalsGituation]

Hi @jfy133,
Thank you for your quick response. I am @igiraclement's colleague on this project, pleased to introduce myself.

We understand the nf-core community’s wish to resurrect the bactmap pipeline for ecosystem hygiene, and the extent of this wish e.g. the March 2025 hackathon to update it. You are right that the functions of steps performed by the pipeline are not where you will find differences between the visions for a bactmap update and for the proposed plasmodiumhapres. Our view is that differences in application, mission and scope are what make expanding the bactmap pipeline to accommodate our proposed workflow not work (at least not comfortably for our intended user base).

In short, we aim to develop a Nextflow-native framework, of which malaria-specific analyses are first-class components and simplicity and specificity are key features. Integrating our proposed workflow into bactmap would remove these features.

A core application of plasmodiumhapres is rolling it out via training programs which we will deliver to public health institutions, laboratories and researchers in malaria-endemic regions. Our users will therefore need a streamlined workflow designed specifically for malaria genomics, as opposed to a highly configurable pathogen pipeline with many organism-specific options.

Such ease of use, a key advantage of releasing on nf-core, brings downstream benefits in reproducibility (easy first run=easy for others to run in exactly the same way). In practice, the user base will be able to pool standardised reports with drug recommendations to track and deal with antimalarial drug resistance in Plasmodium using samples from its Anopheles mosquito vector. We then envision that other users would be easily able to find the pool of reports by searching or making requests for “plasmodiumhapres reports”. Additionally, using the bactmap pipeline for this user base is also likely to introduce a barrier between potential users and the pipeline, since they would be asked to find a pipeline with a name referring to another kingdom of organisms.

To address the remainder of your response:

“The metromap seems to drop a lot of tools (e.g. Medaka), is that expected?

Addressing Medaka specifically, this tool has been superseded by clair3 and pepper (for samples potentially containing mixed infections, at least). Regarding other missing text from the metromap diagram (e.g. MultiQC, samtools), these are only omitted from the visual and have not been dropped from the pipeline. This prototype metromap diagram was produced by @BilalsGituation in order to “clean up” the visualisation, following the philosophy “Simple workflow schematics help outline the main functionality and steps of a pipeline.” the opening sentence of nf-core’s branding guidelines on workflow schematics. Special emphasis on "outline" and "main steps". (Available at https://nf-co.re/docs/community/brand/workflow-schematics)

"if I base the evaluation on the metromap, this still does not seem to be Malaria specific other than the inclusion of PHARE"

Another malaria-specific aspect of the proposed pipeline is this pictured maintenance loop. The loop communicates that the pipeline is underpinned by public health specialists' use of the outputs and their field efficacy to inform the compositional haplotype profiling and/or tree building steps in subsequent pipeline versions.

"(which looks to be a snakemake pipeline, is that correct?)"

Yes, correct. PHARE is indeed an important inspiration and currently exists as a Snakemake-based workflow, but we are planning to extend what PHARE does, rather than simply wrap it inside another pipeline.

"With such a large team (Which is great already!) it would realy help to give the existing bactmap pipeline some TLC (e.g. updating the code base, adding support for modern long read data etc.)."

Basing my understanding on its metromap, the existing bactmap pipeline (https://nf-co.re/bactmap/1.0.0/) shares BWA-MEM and IQTree steps with our proposed pipeline. bactmap would need a bit of overhauling in terms of updating and fitting to our purpose. As a first example, fastp is not suited to removing biological contamination (https://open.bioqueue.org/home/knowledge/showKnowledge/sig/fastp; e.g. the Anopheles mosquito vector) but it is a totally fine cleanup tool when this is not required.

Also, we do not want to give the users the option to sketch (downsample) the input batch in this particular genomic surveillance context. The report should use the haplotype composition results to make time-and-place-specific drug recommendations. Allowing pipeline users to trade off accuracy for reduced resource usage by downsampling batches could cause them to miss rare compositions of pathogen haplotypes, and this could have harmful public health consequences.

We would like to include this updated metromap in our response:

Thank you for reading our response. We anticipate that there is more to discuss but for now we decided to condense our answer to just these concepts so as not to be overwhelmingly extensive.

We look forward to continuing this conversation.

[LouisLeNezet]

Hi,

From the proposed workflow and the justification provided, I still do not see sufficient evidence that this should be implemented as a malaria-specific nf-core pipeline.

Your arguments seems to mostly be around usability, configuration, naming, and target audience. While these are important considerations, they do not in themselves, from my point of view, justify creating a separate pipeline within the nf-core ecosystem. nf-core has generally aimed to separate pipeline functionality from organism-specific configuration wherever possible. I agree however that bactmap would no more be the most adapted named for the pipeline, but this could be fixed for a more general name.

The examples given (e.g. disabling downsampling, replacing fastp with a host-decontamination approach, selecting malaria-specific references, enabling haplotype profiling) appear to be configuration or workflow choices that could be activated through a dedicated profile, for example: -profile plasmodium. This would preserve a streamlined user experience for malaria users while avoiding duplication of a largely overlapping workflow.

The following justification is also a bit vague:

Another malaria-specific aspect of the proposed pipeline is this pictured maintenance loop. The loop communicates that the pipeline is underpinned by public health specialists' use of the outputs and their field efficacy to inform the compositional haplotype profiling and/or tree building steps in subsequent pipeline versions.

Could you clarify how this feedback loop would be implemented within the pipeline itself? As described, it sounds more like a governance, maintenance, or curation process than a workflow component. It is also unclear why such a process would be specific to malaria, as many pathogen surveillance workflows evolve based on expert interpretation of outputs and field observations.