Confused about requirements for speed

[gabedol]

Hi, I'm trying to run colabfold_search locally but my cluster doesn't have SSD, which seems to significantly slow down colabfold_search because of the many I/O calls. How much RAM is required to load in all of the data and skip repeated I/O entirely? Does this differ between GPU and CPU versions of colabfold_search?

[milot-mirdita]

CPU needs about 1-1.5TB RAM to keep everything off disk. GPU slightly more than 48GB VRAM, however it usually runs fine on one L40S.

[gabedol]

Thank you so much milot! So as long as I have a GPU w/ 48 GB, e.g. H100, I can do it quick even without SSD?

[milot-mirdita]

Yes, an H100 should have no slowdowns. You can use the gpuserver to keep the databases in VRAM.

[gabedol]

I'm trying to make MSAs for ~90 sequences (./path_to_fasta). Is this an expected ETA? I'm running on an h100.

colabfold_search ./path/to/fasta ../msa-databases ../msas --gpu 1 --gpu-server 1 --db-load-mode 2

INFO:colabfold.mmseqs.search:Running mmseqs createdb ../msas/query.fas ../msas/qdb --shuffle 0 --dbtype 1
../msas/qdb exists and will be overwritten
createdb ../msas/query.fas ../msas/qdb --shuffle 0 --dbtype 1

Converting sequences

Time for merging to qdb_h: 0h 0m 0s 8ms
Time for merging to qdb: 0h 0m 0s 22ms
Database type: Aminoacid
Time for processing: 0h 0m 0s 288ms
INFO:colabfold.mmseqs.search:Skipping search because ../msas/res.dbtype already exists
INFO:colabfold.mmseqs.search:Skipping mvdb because ../msas/prof_res.dbtype already exists
INFO:colabfold.mmseqs.search:Skipping lndb because ../msas/prof_res_h.dbtype already exists
INFO:colabfold.mmseqs.search:Running mmseqs expandaln ../msas/qdb ../msa-databases/uniref30_2302_db.idx ../msas/res ../msa-databases/uniref30_2302_db.idx ../msas/res_exp --db-load-mode 2 --threads 64 --expansion-mode 0 -e inf --expand-filter-clusters 1 --max-seq-id 0.95
expandaln ../msas/qdb ../msa-databases/uniref30_2302_db.idx ../msas/res ../msa-databases/uniref30_2302_db.idx ../msas/res_exp --db-load-mode 2 --threads 64 --expansion-mode 0 -e inf --expand-filter-clusters 1 --max-seq-id 0.95

Index version: 16
Generated by: 18.8cc5c
ScoreMatrix: VTML80.out
Index version: 16
Generated by: 18.8cc5c
ScoreMatrix: VTML80.out
[=> ] 3.04% 67 eta 11h 19m 27s

nvidia-smi shows mmseqs servers running on the gpu:

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 391741 C python 910MiB |
| 0 N/A N/A 465234 C mmseqs 39878MiB |
| 0 N/A N/A 465235 C mmseqs 9850MiB |
+---------------------------------------------------------------------------------------+