Maximize multi GPU utilization

[amirhp110]

What parameter needs to be changed to maximize the multi GPU utilization.
I did increase the batch size but didn't make any difference. On my 8 x A100 GPU (40 GB) system only 5 GB of each GPU is being used and only 36% gpu utilization. Also not seeing if we are using the mixed precision but that might be my way of checking and the tool report accuracy.

[lucasjinreal]

[amirhp110]

Thanks a lot for the reply,
Did you actually try this and you've seen all GPUs are being utilized at least above 80 percent?
I have to pay to try this, so please let me know your experience.

[lucasjinreal]

how much do u want to pay

[amirhp110]

My main point of my question was if you have already tried a multi GPU system and if you are seeing all GPU are utilizing above 80% or not?

[amirhp110]

I just tried and got the following error; I'll try to fix my local copy of the code as suggested:

RuntimeError: torch.nn.functional.binary_cross_entropy and torch.nn.BCELoss are unsafe to autocast.
Many models use a sigmoid layer right before the binary cross entropy layer.
In this case, combine the two layers using torch.nn.functional.binary_cross_entropy_with_logits
or torch.nn.BCEWithLogitsLoss. binary_cross_entropy_with_logits and BCEWithLogits are
safe to autocast.