Skip to content

Multi GPU and Multi Node

afiaka87 edited this page Apr 12, 2021 · 4 revisions

Distributed Backends

We have two backends you may use to run on a multi-GPU setup.

Deepspeed

Horovod

https://github.com/lucidrains/DALLE-pytorch/wiki/Horovod-Installation