Cannot reproduce the PPL on One Billion Words

[XuezheMax]

For the experiments of language model (LM) on One Billion Words, the final test PPL with Adam and RAdam are around 41 and 40, respectively, worse than the numbers reported in the paper (36.9 for Adam and 35.7 for RAdam).
Github version: 5716b3e

[LiyuanLucasLiu]

Thanks for reaching out! I'll look into this next week.