What is the purpose of the pretrain step?

As I asked in the title, I am a little bit confused about the step pretrain. I thought it was an end-to-end trainning, and I went through your paper, I didn't find any clue about it. Could you please give me more details about your design of this part? Thanks.