How long does the entire RL training take?

[zihui-debug]

Nice work! How many GPUs are required for RL based on your data and training configuration, and how long it will take to complete the training?

[LanXingXuan]

Hi @zihui-debug,

Thank you for your interest in our work!

• Our setup: We used 32 H200 GPUs, and the RL training took approximately 4 days to complete. (~4 steps per hour)

• Minimum setup: The same RL training configuration can be run on a minimum of 8 H200 GPUs. With this setup, it will take about 8 days to finish. (~2 steps per hour)

Hope this helps! Let us know if you have any other questions.