|
Hi all, Currently I'm trying to quantize Qwen 7B model so it can fit into Jetson Orin Nano. The previous repo works on target side but my desktop's Nvidia seems too new (RTX 5060 Ti 16GB). So, it won't work. However, this repo can works on my desktop but I'm not sure if it's going to work on Orin according to the guide. Perhaps any other idea? Thank you! |
Answered by
nvluxiaoz
Feb 5, 2026
Replies: 1 comment
|
Thanks for the discussion! We are planning to be compatible with Jetpack 6.2 series in our next releases. If your Jetson Orin Nano is such platform we could support it. The same code should work on your desktop GPU and Jetson if the precision is int4 or fp16. (Orin does not support fp8 or fp4) |
0 replies
Answer selected by
Thor-x86
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks for the discussion! We are planning to be compatible with Jetpack 6.2 series in our next releases. If your Jetson Orin Nano is such platform we could support it. The same code should work on your desktop GPU and Jetson if the precision is int4 or fp16. (Orin does not support fp8 or fp4)