Description of the bug
bin/add_sh_to_taxonomy.py (used by --addsh to add UNITE species-hypothesis info onto DADA2's taxonomy table) determines which columns are taxonomic ranks with:
rank_cols = [
c for c in taxtable.columns
if c not in ("ASV_ID", "sequence", "confidence") and not c.endswith("_confidence")
]
This doesn't exclude Species_exact, written by DADA2_ADDSPECIES whenever assignTaxonomy's own taxlevels already reach species (the default, unless --skip_dada_addspecies is set). When --addsh and addSpecies are both active, Species_exact gets counted as a rank column, which throws off num_ranks:
- the
SH column is inserted one position later than intended (after Species_exact instead of after Species)
- the per-rank values copied from the UNITE SH lookup (
tax[1 : num_ranks + 1]) are misaligned by one column
Same root cause as the bug just fixed in bin/parse_dada2_taxonomy.r in #1062 (a hand-maintained non-rank-column exclusion list missing a column) -- found while reviewing that PR, but this is a separate script on a separate code path (--addsh), unrelated to #1062's own scope (DADA2 multi-database consolidation), so filing separately rather than folding the fix in there.
Possibly compounding this: existing --addsh test coverage (conf/test_pacbio_its.config) runs with --skip_dada_addspecies, specifically because addSpecies fails on those ITSx-extracted sequences for unrelated reasons -- so --addsh combined with addSpecies output doesn't currently have any test coverage at all.
Steps to reproduce
Run with --addsh and default addSpecies (i.e. without --skip_dada_addspecies) against a reference database whose taxlevels reach species, so assignTaxonomy already populates Species and DADA2_ADDSPECIES renames/duplicates it into Species_exact. Compare the SH-augmented taxonomy table's column order and values against the un-augmented table -- the SH column and the UNITE-lookup values should be shifted one column later than intended.
Suggested fix
Exclude Species_exact from rank_cols, e.g. by matching on suffix (_exact, mirroring the fix applied to bin/parse_dada2_taxonomy.r in #1062) rather than adding one more literal column name.
Description of the bug
bin/add_sh_to_taxonomy.py(used by--addshto add UNITE species-hypothesis info onto DADA2's taxonomy table) determines which columns are taxonomic ranks with:This doesn't exclude
Species_exact, written byDADA2_ADDSPECIESwheneverassignTaxonomy's own taxlevels already reach species (the default, unless--skip_dada_addspeciesis set). When--addshand addSpecies are both active,Species_exactgets counted as a rank column, which throws offnum_ranks:SHcolumn is inserted one position later than intended (afterSpecies_exactinstead of afterSpecies)tax[1 : num_ranks + 1]) are misaligned by one columnSame root cause as the bug just fixed in
bin/parse_dada2_taxonomy.rin #1062 (a hand-maintained non-rank-column exclusion list missing a column) -- found while reviewing that PR, but this is a separate script on a separate code path (--addsh), unrelated to #1062's own scope (DADA2 multi-database consolidation), so filing separately rather than folding the fix in there.Possibly compounding this: existing
--addshtest coverage (conf/test_pacbio_its.config) runs with--skip_dada_addspecies, specifically because addSpecies fails on those ITSx-extracted sequences for unrelated reasons -- so--addshcombined with addSpecies output doesn't currently have any test coverage at all.Steps to reproduce
Run with
--addshand default addSpecies (i.e. without--skip_dada_addspecies) against a reference database whose taxlevels reach species, soassignTaxonomyalready populatesSpeciesandDADA2_ADDSPECIESrenames/duplicates it intoSpecies_exact. Compare theSH-augmented taxonomy table's column order and values against the un-augmented table -- theSHcolumn and the UNITE-lookup values should be shifted one column later than intended.Suggested fix
Exclude
Species_exactfromrank_cols, e.g. by matching on suffix (_exact, mirroring the fix applied tobin/parse_dada2_taxonomy.rin #1062) rather than adding one more literal column name.