Following the GPT Pretraining section in the Megatron-LM repo, we are able to successfully train a model using Megatron-LM
I saw pointers on how to convert from HF to nemo. For example, this conversion script convert_llama_hf_to_nemo.py
However I did not see any examples of converting a ckpt saved using the Megatron-LM to nemo format . Are there any examples for this?
p.s. I am thinking of doing this conversion to nemo, so I can use tools like Nemo-Aligner for post-training
Following the GPT Pretraining section in the
Megatron-LMrepo, we are able to successfully train a model usingMegatron-LMI saw pointers on how to convert from HF to
nemo. For example, this conversion script convert_llama_hf_to_nemo.pyHowever I did not see any examples of converting a ckpt saved using the
Megatron-LMtonemoformat . Are there any examples for this?p.s. I am thinking of doing this conversion to
nemo, so I can use tools likeNemo-Alignerfor post-training