GRPO dataset(s)

[ocramz]

in the README, the example for launching a RL job uses a specific dataset:

AI-MO/NuminaMath-TIR

• Is there a specific reason for this?
• would it be good to have additional datasets of this "reasoning" form, possibly from domains other than math?

[qgallouedec]

Is there a specific reason for this?

It's a nice dataset that have verifiable answer. But technically, you can use any verifiable data.

would it be good to have additional datasets of this "reasoning" form, possibly from domains other than math?

Indeed, we've a lot a of issues opened to suggest other domains. There's a lot to explore here, for sure.

[ocramz]

thank you @qgallouedec ! I am actually working on such a dataset, for a program execution domain.
Do we have a preferred dataset format in this project?

[qgallouedec]

Yes, for GRPO, you should use "prompt-only" dataset, see https://huggingface.co/docs/trl/dataset_formats. Nite that it's named "prompt-only", but in case of GRPO, you probably want to add other columns. See https://huggingface.co/docs/trl/grpo_trainer#using-a-custom-reward-function