RLHF Tracker

[SalmanMohammadi]

Tasks

Preview Give feedback

1. Full-finetune DPO single device recipe #2082
   enhancement +1
2. Full DPO Distributed #2275
   CLA Signed +1
3. [Docs] Write PPO and DPO recipe docpages #1262
   documentation +1
   
4. R1-Style distributed GRPO #2326
   CLA Signed +1
5. PPO tutorial/deep dive
6. DPO tutorial/deep dive
7. Multimodal support for DPO
8. Sample packing for preference datasets
9. General support for classification models for PPO and reward modelling
10. Reward modelling recipe
11. E2E RLHF blogpost
12. Full-finetune Distrbuted PPO Recipe

These are some RLHF-related features we'd like to see in torchtune. If you're interested in working on any of these, please open a separate issue for the task and recieve approval from a maintainer before opening a PR.