same chosen and rejected dpo data #592
Description
Hi,
I have noticed that in your dpo_hotpotqa_workforce data, chosen and rejected decomposed planning tasks are exactly similar. Out of 220 total questions/rows, 144 have exactly same decomposed tasks. generally, in dpo training, chosen and rejected items are different.
Owl related all dpo data have identical chosen & rejected decomposed plans.
Is this by mistake and if not, what is the logic behind this if model sees exactly same chosen and rejected subtasks.
could you please update correct dpo data.
https://huggingface.co/datasets/owl-agent/dpo_hotpotqa_workforce
https://huggingface.co/datasets/owl-agent/dpo_math_workforce
https://huggingface.co/owl-agent
Dataset was uploaded by one of the author on OWL paper.
Looking forward to hear.
