-
Notifications
You must be signed in to change notification settings - Fork 2
Update MM docs #498
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Update MM docs #498
Conversation
docs/mm/nemo_2/in-framework.md
Outdated
|
|
||
| 1. Follow the steps on the [Generate A NeMo 2.0 Checkpoint page](gen_nemo2_ckpt.md) to generate a NeMo 2.0 multimodal checkpoint. | ||
|
|
||
| 2. In a terminal, go to the folder where the ``qwen2_vl_3b`` is located. Pull and run the Docker container image using the command shown below. Change the ``:vr`` tag to the version of the container you want to use: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The model generated in gen_nemo2_ckpt.md and the one used here qwen2_vl_3b do not match. Could you please to use the one generated here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@oyilmaz-nvidia I've update the section in docs.
Export-Deploy/docs/mm/nemo_2/gen_nemo2_ckpt.md
Lines 21 to 42 in f0799fe
| ## Generate Qwen VL Checkpoint (for In-Framework Deployment) | |
| This checkpoint is used for in-framework deployment examples. | |
| 3. Run the following Python code to generate the NeMo 2.0 checkpoint: | |
| ```python | |
| from nemo.collections.llm import import_ckpt | |
| from nemo.collections import vlm | |
| from pathlib import Path | |
| if __name__ == '__main__': | |
| # Specify the Hugging Face model ID | |
| hf_model_id = "Qwen/Qwen2.5-VL-3B-Instruct" | |
| # Import the model and convert to NeMo 2.0 format | |
| import_ckpt( | |
| model=vlm.Qwen2VLModel(vlm.Qwen25VLConfig3B(), model_version='qwen25-vl') | |
| source=f"hf://{hf_model_id}", # Hugging Face model source | |
| output_path=Path('/opt/checkpoints/qwen25_vl_3b') | |
| ) | |
| ``` |
No description provided.