Trajectory split in ReasonFlux-PRM

[Niax23]

Hi authors, thank you for your excellent work of ReasonFlux-PRM!
Would you please provide some details on how to segment trajectories into several steps and how the alignment between the trajectory and the model response is established?

[akadaii]

Hi there, I got the same question. Have you worked out with it?

[jiaruzouu]

Hi, thank you for your interest in our work. Regarding the splitting of thinking segments, a common and straightforward approach is to split them using the "\n\n" delimiter.

Regarding the alignment between intermediate thinking trajectories and model responses: Theoretically, the number of thinking steps and responses should be the same, since the final model response is generated conditioned on the preceding thinking trajectories. In practice, however, we occasionally observe a small mismatch in the number of steps between the two. Most commonly, the number of thinking trajectories is slightly larger than that of the final response. In such cases, we identify the step that causes the discrepancy and merge the extra step with the preceding one to ensure alignment with the corresponding part of the final model response.

We hope this could address your questions. Please let me know if you have any other questions! Thanks again for the interest of our work!