Description of feature
Phylosearch mode runs one HMM profile per row of --phylosearch_input, and today the pipeline places, grafts, classifies and heat-trees each profile's hits independently (workflows/phyloplace.nf -> FASTA_NEWICK_EPANG_GAPPA, keyed throughout by the row's target/meta.id).
This is a problem when several profiles are deliberately hierarchical and share one reference tree: e.g. RNR class profiles NrdA/NrdJ/NrdD plus subclass profiles NrdAe/NrdAg/NrdAr. The subclass profiles exist specifically to steer sequences away from false positives at class level (NrdAr hits that would otherwise be picked up, and misclassified, by NrdJ), but the reference tree is built at class level (NrdA), so NrdAe/NrdAg/NrdAr all place onto the same reference tree as NrdA itself. Today that produces one grafted tree / classification / heat tree per profile, i.e. several trees for hits that conceptually belong on one.
Classification output isn't really a problem this way, since the per-profile TSVs can just be concatenated afterwards. Grafted trees (and arguably heat trees) are the real pain point: several trees where one, containing all placements, would be much more useful.
Proposal
gappa examine graft and gappa examine assign both already accept multiple jplace files in one invocation (--jplace-path is a list, can point at several files or a directory) and produce one merged tree / classification when the jplace files reference the same underlying tree. Confirmed via gappa examine graft --help / gappa examine assign --help (v0.8.0, the version already vendored via modules/nf-core/gappa/examinegraft and .../examineassign) -- this isn't a new gappa capability, just an input pattern the pipeline doesn't currently use.
So the idea is: in addition to today's per-profile grafted tree / classification / heat tree, add a second, per-reference-tree summarisation step that concatenates the jplace files of every profile sharing a reference tree and runs gappa examine graft / examine assign (and maybe examine heattree) over that concatenation once, producing one joint output per distinct reference tree.
Open questions to settle during implementation, not blocking the idea itself:
- Grouping key.
refphylogeny itself could be the natural grouping key (same file = same tree), but the user suggested an explicit optional reftreename column in assets/schema_phylosearch_input.json instead, so trees can be named meaningfully in the summary output rather than by an internal file path. Rows that don't set it would presumably keep only today's per-profile behaviour (opt-in, not a breaking change).
- Taxonomy consistency.
gappa examine assign takes a single --taxon-file; profiles grouped under one reftreename need a consistent taxonomy file for that to make sense. Needs a decision on what happens if grouped rows disagree (error out vs. take the first vs. require it be blank on all but one row).
- Naming/publishing of the new joint outputs alongside the existing per-profile
gappa/*.graft.*.newick / *.taxonomy.* / *.heattree.* files (docs/output.md "Summary" section).
Environment
- Nextflow version: n/a (design proposal)
- Pipeline version: 2.2.0dev
Description of feature
Phylosearch mode runs one HMM profile per row of
--phylosearch_input, and today the pipeline places, grafts, classifies and heat-trees each profile's hits independently (workflows/phyloplace.nf->FASTA_NEWICK_EPANG_GAPPA, keyed throughout by the row'starget/meta.id).This is a problem when several profiles are deliberately hierarchical and share one reference tree: e.g. RNR class profiles
NrdA/NrdJ/NrdDplus subclass profilesNrdAe/NrdAg/NrdAr. The subclass profiles exist specifically to steer sequences away from false positives at class level (NrdArhits that would otherwise be picked up, and misclassified, byNrdJ), but the reference tree is built at class level (NrdA), soNrdAe/NrdAg/NrdArall place onto the same reference tree asNrdAitself. Today that produces one grafted tree / classification / heat tree per profile, i.e. several trees for hits that conceptually belong on one.Classification output isn't really a problem this way, since the per-profile TSVs can just be concatenated afterwards. Grafted trees (and arguably heat trees) are the real pain point: several trees where one, containing all placements, would be much more useful.
Proposal
gappa examine graftandgappa examine assignboth already accept multiple jplace files in one invocation (--jplace-pathis a list, can point at several files or a directory) and produce one merged tree / classification when the jplace files reference the same underlying tree. Confirmed viagappa examine graft --help/gappa examine assign --help(v0.8.0, the version already vendored viamodules/nf-core/gappa/examinegraftand.../examineassign) -- this isn't a new gappa capability, just an input pattern the pipeline doesn't currently use.So the idea is: in addition to today's per-profile grafted tree / classification / heat tree, add a second, per-reference-tree summarisation step that concatenates the jplace files of every profile sharing a reference tree and runs
gappa examine graft/examine assign(and maybeexamine heattree) over that concatenation once, producing one joint output per distinct reference tree.Open questions to settle during implementation, not blocking the idea itself:
refphylogenyitself could be the natural grouping key (same file = same tree), but the user suggested an explicit optionalreftreenamecolumn inassets/schema_phylosearch_input.jsoninstead, so trees can be named meaningfully in the summary output rather than by an internal file path. Rows that don't set it would presumably keep only today's per-profile behaviour (opt-in, not a breaking change).gappa examine assigntakes a single--taxon-file; profiles grouped under onereftreenameneed a consistent taxonomy file for that to make sense. Needs a decision on what happens if grouped rows disagree (error out vs. take the first vs. require it be blank on all but one row).gappa/*.graft.*.newick/*.taxonomy.*/*.heattree.*files (docs/output.md"Summary" section).Environment