AGX Thor / L4T r39 / CUDA 13.2 (JetPack 7.2) - Build failures and workarounds

[mrjbj]

Human:

I worked with AI to get jetson-containers cudastack built on AGX Thor r39 and here is a summary of findings we worked through in order to get things built. Posting here in case others find it useful.

AI:

Hardware: NVIDIA Jetson AGX Thor Developer Kit
L4T: R38.4.0 (detected as r39.0 by jetson-containers)
JetPack: 7.2
CUDA: 13.2
Ubuntu: 24.04 aarch64
Docker: docker.io 28.2.2 from Ubuntu repos (not docker-ce)

Successfully built ollama after working through the following issues:

---

Issue 1: Test harness fails on Thor (all tests)
The test runner uses --gpus=all which is not supported on Jetson Thor.
Thor requires --runtime=nvidia instead.
Workaround: ./build.sh ollama --skip-tests all

---

Issue 2: pypi.jetson-ai-lab.io/sbsa/cu132 registry missing common packages
The sbsa/cu132 PyPI index is missing many standard packages including:
pip, psutil, setuptools, packaging, Cython, wheel, uv, nvidia-ml-py, twine, cmake, numpy, huggingface_hub, ollama

This causes uv pip install calls to fail with "not found in package registry".

Fix: Add --extra-index-url https://pypi.org/simple to all uv pip install calls in:

• packages/build/python/install.sh (lines 57, 63, 64, 73)
• packages/build/cmake/cmake_pip/install.sh
• packages/ml/numeric/numpy/install.sh
• packages/llm/huggingface_hub/Dockerfile
• packages/llm/ollama/docker_files/tmp/OLLAMA/build.sh
• And many others — bulk fix with:

grep -rl "uv pip install" packages/ --include="install.sh" | xargs sed -i \
  '/extra-index-url/!s|uv pip install |uv pip install --extra-index-url https://pypi.org/simple |g'

grep -rl "uv pip install" packages/ --include="Dockerfile" | xargs sed -i \
  '/extra-index-url/!s|uv pip install |uv pip install --extra-index-url https://pypi.org/simple |g'

---

Issue 3: TensorRT download requires authentication
The TensorRT URL in cudastack config points to developer.nvidia.com which requires login.
Additionally, file:// URLs do not work in Docker build context.

Workaround: Download manually from developer.nvidia.com (ARM SBSA CUDA 13.2 TAR), then serve locally:

cd /path/to/downloads
python3 -m http.server 8888
./build.sh ollama --skip-tests all \
  --build-arg TENSORRT_URL:"http://localhost:8888/TensorRT-10.16.0.72.Linux.aarch64-gnu.cuda-13.2.tar.gz"

Note: downloaded file was cuda-13.2 variant, not cuda-13.1 as specified in config.

---

Performance note:
Using the pre-built dustynv/ollama:0.6.8-r36.4-cu126-22.04 image on Thor gives only ~19 t/s on llama3.2:3b Q4_K_M. Published benchmarks show ~150 t/s expected. Building from source with the correct r39/cu132 stack should resolve this.

Happy to test fixes or submit a PR for the bulk pypi fallback change.

[johnnynunez]

use your own index

sudo bash install.sh --pypi

[mrjbj]

I wasn't really sure if I should have posted this or not as frankly, I didn't understand a lot of the nuances involved. I was grinding through it all interactively, with AI helping me resolve along the way. Please advise if this sort of feedback is helpful to the overall jetson-containers initiative. I have some similar learnings I could share from AI assisted debugging efforts to get vLLM inference engine working on Thor AGX L4T r39 that I could share as well but don't want to pollute the repo if not helpful. There were a lot of issues to work though and i got it working, but AI advised on most of the fixes, so might not be considered appropriate to share in this way. Let me know if useful. Thanks.

[johnnynunez]

I wasn't really sure if I should have posted this or not as frankly, I didn't understand a lot of the nuances involved. I was grinding through it all interactively, with AI helping me resolve along the way. Please advise if this sort of feedback is helpful to the overall jetson-containers initiative. I have some similar learnings I could share from AI assisted debugging efforts to get vLLM inference engine working on Thor AGX L4T r39 that I could share as well but don't want to pollute the repo if not helpful. There were a lot of issues to work though and i got it working, but AI advised on most of the fixes, so might not be considered appropriate to share in this way. Let me know if useful. Thanks.

dusty was retired, so jetson-containers is not associated with nvidia. Use your own pypi to get a server to build.
Anyways, Thor is SBSA. You can use public wheels. Orin it will be around end of April with JP7.2

[johnnynunez]

now dusty containers has own pypi if you want to create your personal one

git clone https://github.com/dusty-nv/jetson-containers
cd jetson-containers
bash jetson-containers/install.sh --pypi 

it creates everything for you automatically
then

source .env
jetson-containers build pytorch