Skip to content

README.md

Latest commit

 

History

History
145 lines (117 loc) · 8.64 KB

README.md

File metadata and controls

145 lines (117 loc) · 8.64 KB
Raw

ePlacer

ePlacer is a taxonomic classification tool that uses deep-learning approaches to incorporate both sequence information and biogeographic information into taxonomic assignment of DNA sequences.

Why use ePlacer

The machine learning architecture of ePlacer enables powerful prediction beyond sequence-only classification tools (e.g. sequence alignment with blast or naive-bayes classifiers) by directly incorporating additional data into the probabalistic estimate of taxonomy, specifically developed for metabarcoding data. This novel applciation of deep-learning is immensely useful, as there can be many cases in metabarcoding data where two reference species have 100% sequence overlap, but distinct geographic ranges. This tool discriminates these cases and provides additional data for downstream taxonomic curation. Due to this, ePlacer provides enhanced interoperability between metabarcoding datasets.

Currently, ePlacer offers pre-trained models for two popular metabarcoding regions: the MiFish and the ecoPrimer, or Riaz, marker gene regions. For these two regions, ePlacer offers the following benefits:

  • Interoperability. ePlacer is trained on global datasets, allowing for direct comparison between metabarcoding datasets, regardless of geographic region.
  • Portability. ePlacer has pre-trained models available for both MiFish and Riaz marker gene regions containerized and available for out-of-the-box use
  • Increased Accuracy. The ePlacer model architecture provides increased accuracy, precision, and recall as compared to blast, Naive-Bayes, or least common ancestor approachers
  • Trainability In addition to the two provided barcodes, this code repository provides tools for training new models.

For other barcode regions, there will be significant advantages with the training of new models. If you are interested in training a new model for ePlacer, please do not hesitate to reach out!

Installation

Users can install the current version of ePlacer with conda.

conda install bioconda::eplacer

or pip

pip install eplacer

Using ePlacer for classification

The ePlacer taxonomic assignment tool can be run two ways: natively (through the ePlacer CLI or API) or with a QIIME2 plugin. Here, the documentation will be detailing the native usage. Details on usage of the QIIME2 plugin can be found in the linked git repository.

ePlacer taxonomically classified ASV sequences using two distinct types of information:

  • Sequence information (inferred from ASVs)
  • Biogeography (inferred from sample metadata and count tables)

Although not strictly required for assignment, blast results are also used to automatically check "solvable" taxonomic assignments and resolve them more accurately as an automated curation step.

Using this information, ePlacer generates a raw confidence of presence across all possible taxonomic labels.

In order to run classification with ePlacer, four data files are required. Properly formatted examples can be seen here:

  • A fasta file of ASVs
>ASV1
CCGTAAACTTAGATAAATTAGTACAACAAATATCGGCCCGGGAACT
>ASV2
CGGTAAACTTAGATATATTAGTACAACAAATATCGGCCCGGGAACT
>ASV3
CGGTAAACTTAGATATATTAGTACAACAAATATCGGCCCGGGAACT
  • A geography metadata file
#SampleID	Latitude	Longitude
Sample1	39.645946	-71.746641
Sample2	39.645946	-71.746641
  • A count table
#OTU ID	Sample1	Sample2
ASV1	15	0
ASV2	5	22
ASV3	0	10
  • blast data output (generated with -outfmt "6 qseqid sseqid pident evalue length qlen slen qstart qend sstart send sseq"). Truncated example:
ASV1	SubjectRef_A	100.00	1.45e-45	98	98	98	1	98	...
ASV2	SubjectRef_B	99.00	2.12e-42	98	98	98	1	98	...
ASV3	SubjectRef_C	100.00	1.45e-45	98	98	98	1	98	...

Acquiring pre-trained models.

Pre-trained models can be acquired from Zenodo (doi:10.5281/zenodo.20835759). Currently, only 12S-V5 ecoprimer and mifish primers are available, but others will be created and stored in the future. If you develop your own model, please don't hesitate to reach out.

Natively trained models contain directories of information and can be obtained in the following manner:

wget https://zenodo.org/records/20835759/files/mifish\ (2).tar.gz
tar -xzf mifish\ (2).tar.gz
wget https://zenodo.org/records/20835759/files/riaz_12SV5.tar.gz
tar -xzf riaz_12SV5.tar.gz

Running Classification with Pre-trained models

For users that have generated their own models or have downloaded a model from the zenodo repository, use the following code:

eplacer run-model --fasta <fasta path> \
                  --counts <count matrix> \
                  --geoData <geoData path> \
                  --confidence <threshold> \
                  --model <model path> \
                  --maskrate 0  \
                  --blast <blast results>

This will generate two output files in ./result/models/

  • bestGeoPredict.tsv: a file containing a summary of the best predictions made by the model
  • totalGeoPredict.yaml: a yaml file detailing all possible results for every ASV to facilitate independent analysis.
A Note on the output

The ePlacer output, bestGeoPredict.tsv has a large number of columns and data. By default, ePlacer bootstraps the assignment across possible geographic locations and applied the automated curation protocol.

See descriptions detailed below:

Column Name Description
ASV Name of the ASV
Curated Taxa Assignment of ePlacer after the automated curation protocol
Curated Taxa Level Taxonomic level of the final assignment
Predicted Taxa Raw assignment of ePlacer
Predicted Taxa Level Taxonomic level of the raw assignment
All Top Scoring hits Semicolon delimited list of taxonomy that exceeded a minimal probability threshold during inference (>0.01)
Top Scoring Mean Prob Semicolon delimited list of mean probalities of Top Scoring Hits
Top Scoring Std Dev Probs Semicolon delimited list of probality standard deviation of Top Scoring Hits
Top Scoring Geo Semicolon delimited list of Top Scoring Hits showing proportion of geographic locations assessed that fall in the known range based on the reference database
Assignment Note A note on the assignment, usually referring to 100% blast hits and known geographic ranges

Although information dense, this output allows the end use to better evaluate the overall perforamnce of the model.

Training new ePlacer models

Training new ePlacer models is very simple! All that is required is an aligned fasta file for the barcode of interest (containing all available references of interest), a flat taxonomy file (tab separated with fields 'Accession', 'Kingdom', 'Phylum', 'Class', 'Order', 'Family', 'Genus', 'Species'), and a reference file for biogeography (currently, eplacer supports the OBIS csv download). ePlacer also supports custom references for biogeography, formatted as follows:

#Species	Latitude	Longitude
SpeciesLabelA	39.645946	-71.746641
SpeciesLabelB	39.645946	-71.746641
A note on biogeography

Repeated entries for different species are supported to develop a full understanding of species distribution.

To run the training, use the following:

eplacer train-model --fasta <alignment file> \
            --taxa <taxonomy file> \
            --out <output directory> \
            --taxlevel SPECIES \
            --geoData <geography data> \
            --augments <INT> \
            --maskrate <Proportion> \
            --threads 1

Training results in two output: a directory containing all the model files required for running (plus a few extra useful ones) and a flat file of statistics for the model.

=============================================

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.