can I offload vram to 3 gpus?

[JemZhu]

can I offload vram to 3 gpus? I have 6 V100 16G gpus on the server. The flux.2 dev model is too large to offload on 2gpus. Is there a way to offload vram to 3 or more gpus? thanks.

[an80sPWNstar]

You can! I just tried it. Here's what I did. I am SUPER HAPPY :)

[tjengbudi]

Any walk trough or tips?

…

On Tue, Jan 6, 2026, 05:03 an80sPWNstar ***@***.***> wrote: *an80sPWNstar* left a comment (pollockjj/ComfyUI-MultiGPU#156) <#156 (comment)> You can! I just tried it. Here's what I did. I am SUPER HAPPY :) image.png (view on web) <https://github.com/user-attachments/assets/3474aa91-5bf2-4ea9-878d-3e09f279241c> image.png (view on web) <https://github.com/user-attachments/assets/57e2171f-56ef-4140-a37e-e1d2ec60ba61> — Reply to this email directly, view it on GitHub <#156 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXGEPYE47DGQPFKM4QJSL34FLNSHAVCNFSM6AAAAACPQE5OCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMJSGIZTSOBTGU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[an80sPWNstar]

Are you able to see the images I uploaded? It shows exactly how I got it to work. Find out how big the model files are and do the math on how much you can fit on each card; follow the same pattern I showed in the image and it will work. Lemme know if it doesn't.

[JemZhu]

It works! Thank you!! Now with the nvlink between 6 V100s , the speed is almost same as a single V100. Now I have a v100 96G. SUPER HAPPY +1, hahaha.

[an80sPWNstar]

Dang! Glad I could help.

[stranger-games]

It works! Thank you!! Now with the nvlink between 6 V100s , the speed is almost same as a single V100. Now I have a v100 96G. SUPER HAPPY +1, hahaha.

Does it actually take advantage of the nvlink and increase the speed of communication of the donor devices?

[JemZhu]

Yes, NVLink provides 100 GB/s low-latency communication between GPUs and allows direct access to the HBM of other V100s. much faster than going through PCIe → CPU → PCIe

[overallbit]

@JemZhu do you have any info if the secondary GPUs power draw runs at max?

[tjengbudi]

@an80sPWNstar can we use GPU with different type? for example rtx 3090, rtx 4060 and rtx 3060 with multigpu?

[an80sPWNstar]

@an80sPWNstar can we use GPU with different type? for example rtx 3090, rtx 4060 and rtx 3060 with multigpu?

Yup! I have a rtx 5060 ti, 5070 ti and a 3090 that I've used all at the same time.

[droplotus]

@an80sPWNstar You can! I just tried it. Here's what I did. I am SUPER HAPPY :)

I'm quite new into this image generation world, so how can you use KSampler on cuda:1? You have compute_device cuda:1, donor cuda:0, if I try that (which works they split memory), my KSampler gives me an error "Can't export tensors on a different CUDA device index. Expected: 1. Current device: 0."

I'm using cuda:1 for Vae + Clip as well.
cuda:0 -> 4070S 12VRam
cuda:1 -> 5060Ti 16VRam