how the weight initialize?

Great work！
Can you give an example or explain how the weight is initialized? I guess it might be the number of tokens or flops? Does the number come from the training phase or the inference phase?