How to switch on the multi-GPU for GRPO Training?

[tjoymeed]

Issue:
OOM errors during GRPO training - Need multi-GPU support for combined VRAM

Problem Description:
I'm encountering Out-of-Memory (OOM) errors while using GRPO Training to train reasoning capabilities similar to DeepSeek R1.

My Question:
How to switch on multi-GPU support for GRPO Training to utilize the combined VRAM across multiple GPUs (e.g., 40GB × 8 cards = 320GB total VRAM)?

Thank you!