Discrepancy between paper and code regarding ReasonFlux-F1 training?

[catundchat]

I am interested in your work and would like to try to reproduce it. In the process, I've some questions:
I found two sft code in this repository, but no RL code found.
In the paper it states that there is reinforcement learning (sampling trajectories, calculating reward functions, preference data pairs, etc.), but that doesn't seem to be the case on this graph? I'd like to ask if the llm(ReasonFlux-F1) was obtained directly after the third step?

[YangLing0818]

I am interested in your work and would like to try to reproduce it. In the process, I've some questions: I found two sft code in this repository, but no RL code found. In the paper it states that there is reinforcement learning (sampling trajectories, calculating reward functions, preference data pairs, etc.), but that doesn't seem to be the case on this graph? I'd like to ask if the llm(ReasonFlux-F1) was obtained directly after the third step?

Thank you for your interest in our work. We would like to clarify that our current paper focuses exclusively on the details of ReasonFlux-Zero. The ReasonFlux-F1 model, which is built upon ReasonFlux-Zero as illustrated in the figure, represents a newly trained version of our model (we may write a new technical report for the F1 model).

At present, our manuscript detailing ReasonFlux-Zero is under review. Once it has been accepted, we plan to release additional components, including the reinforcement learning (RL) part.

[arunraman]

Am curious about this as well. What's the difference between ReasonFlux-Zero and ReasonFlux-F1