Skip to content

[QUESTION] Converting a Megatron-LM ckpt to Nemo #1206

@abgoswam

Description

@abgoswam

Following the GPT Pretraining section in the Megatron-LM repo, we are able to successfully train a model using Megatron-LM

I saw pointers on how to convert from HF to nemo. For example, this conversion script convert_llama_hf_to_nemo.py

However I did not see any examples of converting a ckpt saved using the Megatron-LM to nemo format . Are there any examples for this?

p.s. I am thinking of doing this conversion to nemo, so I can use tools like Nemo-Aligner for post-training

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions