Installing Alpamayo 1.5 on Jetson Thor with JetPack 7.2

[markma12]

I have a JETSON THOR 128 computer with JETPACK 7.2 which has CUDA 13.2. I was unable to run your network on the above computer.
Is there any way to run it on a THOR computer?
Has anyone done this?
Is it possible?

[super-anova]

Hi, thanks for trying our model!

We only tested and validated on Cuda 12 (see also https://github.com/NVlabs/alpamayo1.5#prerequisites) so that's the first thing that stood out to me.

In addition, did you have to build flash-attn from source for ARM?

What specific error did you get?

[UserName-wang]

Hi!
I tried nv-community/Alpamayo-R1-10B on jetson thor with JETPACK 7.2 which has CUDA 13.2. the inference speed is very slow. the model feedback the results after about 20~30 seconds. I build flash-attn from source for ARM.

[super-anova]

Hi @UserName-wang. Thanks for sharing that data point! Inference latency on that time scale is roughly in line with what we would expect from Jetson Thor.

[super-anova]

@UserName-wang Fyi, we released a quantization recipe for Alpamayo 1.5 model (https://github.com/NVlabs/alpamayo-recipes/tree/main/recipes/alpamayo1_5_quant). It may help you reduce the memory footprint and potentially help with latency as well due to the lower precision flops.