[Database] MicroEuk100 v3 has Bacteria on them?

[franlat]

Hello!

I've been using VEBA MicroEuk100 for some analyses latelty, and while it works great. I found some hits to Bacteria coming from the DB. When I check the source_taxonomy.tsv file, I can see there are some bacteria listed there:

EP00049	EukProt	50559	d__Bacteria;p__Bacillota;c__Clostridia;o__Eubacteriales;f__;g__;s__	Clostridia	Eubacteriales						False
EP00693	EukProt	50422	d__Bacteria;p__Pseudomonadota;c__Gammaproteobacteria;o__Alteromonadales;f__Shewanellaceae;g__Shewanella;s__Shewanella sp.	Gammaproteobacteria	Alteromonadales	Shewanellaceae	Shewanella	Shewanella sp.			True
EP01061	EukProt	50226	d__Bacteria;p__Actinomycetota;c__Actinomycetes;o__Mycobacteriales;f__Mycobacteriaceae;g__Mycobacterium;s__Mycobacterium sp. T103	Actinomycetes	Mycobacteriales	Mycobacteriaceae	Mycobacterium	Mycobacterium sp. T103			True
MMETSP1091	MMETSP	1073	d__Bacteria;p__Pseudomonadota;c__Alphaproteobacteria;o__Hyphomicrobiales;f__Nitrobacteraceae;g__Rhodopseudomonas;s__	Alphaproteobacteria	Hyphomicrobiales	Nitrobacteraceae	Rhodopseudomonas				False
MMETSP1389	MMETSP	1073	d__Bacteria;p__Pseudomonadota;c__Alphaproteobacteria;o__Hyphomicrobiales;f__Nitrobacteraceae;g__Rhodopseudomonas;s__	Alphaproteobacteria	Hyphomicrobiales	Nitrobacteraceae	Rhodopseudomonas				False

I also work with EukProt v3, but when I go to their list of genomes, EP00049, EP00693 and EP01061 do not match on the taxonomy. These IDs match to:
EP00049_Salpingoeca_infusionum, EP00693_Jakoba_libera, and EP01061_Rhynchopus_euleeides.

Is this somehow expected?

I built an MMSeqs2 DB with VEBA MicroEuk and used the taxids in source_taxonomy.tsv to give each protein a taxonomy. However, the LCA algorithm has some trouble when it involves genes from these datasets.

[jolespin]

Ah apologies for this. I should have checked but assumed the EukProt and Marine Microbial Eukaryote Transcriptome Sequencing Project (MMETSP) only had eukaryotes. . I have also tried building a MMseqs2 database with MicroEuk100 but was unsuccessful. I believe it was due to the fact that many euks have unconventional taxonomy with clades but the error messages were never too informative so I could never find the source of the error.

Regardless, thank you for reporting this. I can try to clean this up for the next major VEBA release

[franlat]

I don't know about MMETSP, but the error in EukProt might be when translating names to taxid? Since EukProt per se works with UniEuk IDs. The file source_taxonomy.tsv has these mistakes, but identifier_mapping.proteins.tsv doesn't seem to. The organism names look okay.

In any case, I can check later if only these 5 missmatch and let you know.

When building the MMSEQS2 DB I found two issues that I had to fix first in the source_taxonomy.tsv

1. Some organisms, the TARA MAGs, do not have a taxid associated, but they do have taxonomy names. I used those to give them one “manually”. I used Entrez package in Python, but some did not match correctly, and had to edit them myself. e.g., it detects Micromonas as the bacteria Parvimonas, because in the past it was called Micromonas. For some reason, assigns incorrectly all the Micromonas taxid.
2. Some MMETSP entries have duplicated, which made me have duplicated in the mapping file.

They way I built the database was using NCBI's taxdump and making the mapping myself by transferring the organism taxid to their genes, like this:

0007d156e1c6cbc53f215f1bbead607d        5599
0026fd5bbc8c18b442e2ec19fa296800        5599
003ce9f4c7154566ea47295f7aa2868c        5599
004440544f12d9d34977b7654b391285        5599
0055c11ca8953477a94c1a51697908da        5599
006199c2251009fb02c60c1293b735cd        5599
0089d551dacbfb9a72ceebd12da99791        5599
00948d99856afaf664b215fee8b08621        5599
009f4dbfd4237026f2ff6c38f0fce2e9        5599
00a0400706e0ed9ecda5d7b2fea92b6e        5599

[jolespin]

I don't know about MMETSP, but the error in EukProt might be when translating names to taxid? Since EukProt per se works with UniEuk IDs. The file source_taxonomy.tsv has these mistakes, but identifier_mapping.proteins.tsv doesn't seem to. The organism names look okay.

This appears to be the case because I need to expand the lineage outwards from the taxid when the lineage string isn't provided. There must be a few errors in the original database.

Some organisms, the TARA MAGs, do not have a taxid associated, but they do have taxonomy names. I used those to give them one “manually”. I used Entrez package in Python, but some did not match correctly, and had to edit them myself. e.g., it detects Micromonas as the bacteria Parvimonas, because in the past it was called Micromonas. For some reason, assigns incorrectly all the Micromonas taxid.

Ah yes, this is ringing a bell. I recommend taxopy for fuzzy name searches or taxid lookup but the problem with many of these environmental genomes is that the names are ids that might not be deposited in NCBI or if they are the names could be a variation.

Some MMETSP entries have duplicated, which made me have duplicated in the mapping file.

Can you provide a bit more context here?

Also, I'm realizing that all of these massive tables would have huge performance gains w/ i/o as parquet. I'll change these in the update.

[franlat]

Some MMETSP entries have duplicated, which made me have duplicated in the mapping file.

Can you provide a bit more context here?

Yes, of course. In the source_taxonomy.tsv, which links every organism to its taxonomy, there are some organisms duplicated. For example, MMETSP1325 appears 25 times duplicated in the list, and MMETSP1346 appears 16 times. It is the same exact line for all of them, though.

[franlat]

It's been a few months, but I had some free time and manually corrected the bacterial taxIDs. I used the species name provided by EukProt v3 and the MMETSP db. Also removed the duplicated entries. I don't know about the spacing, though. I only needed the first three columns at the end, so it may be a bit off. But perhaps it will be helpful to you for future releases.

source_taxonomy.fixed.tsv