Skip to content

Qwen2.5 export fails with ValueError because _resolve_shard rejects Hugging Face cache symlinks #184

Open
#195
Open
Qwen2.5 export fails with ValueError because _resolve_shard rejects Hugging Face cache symlinks#184
#195
Labels
bugSomething isn't working

Description

@FABallemand
FABallemand
opened on Aug 17, 2026

Describe the bug

Exporting officially supported Qwen2.5 and Qwen2.5-VL checkpoints fails with TensorRT Edge-LLM 0.10.0.

During export, tensorrt-edgellm/checkpoint/loader.py:_resolve_shard resolves the shard symlink to its target in the Hugging Face blobs/ directory and then checks whether the resolved path is inside the model's snapshots/ directory. This check fails for Hugging Face's standard cache layout.

The resulting error is:

XX:XX:XX  ERROR     tensorrt_edgellm.scripts.export: [LLM] Failed to load checkpoint
Traceback (most recent call last):
  File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 286, in _resolve_shard
    resolved.relative_to(base)
  File "/usr/lib/python3.12/pathlib.py", line 682, in relative_to
    raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}")
ValueError: '/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-3B/blobs/51410930d5cf19a998fdb17ef0c46e4d9ace72c97a975a3331395a8a500f5edb' is not in the subpath of '/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-3B/snapshots/3aab1f1954e9cc14eb9509a215f9e5ca08227a9b'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/scripts/export.py", line 1030, in _export_llm
    model = AutoModel.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/model.py", line 543, in from_pretrained
    load_weights(model,
  File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 127, in load_weights
    shard_map = _build_shard_map(model_dir)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 337, in _build_shard_map
    if not os.path.exists(_resolve_shard(model_dir, shard))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 288, in _resolve_shard
    raise ValueError(
ValueError: Shard path 'model-00002-of-00002.safetensors' in checkpoint index escapes model_dir '/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-3B/snapshots/3aab1f1954e9cc14eb9509a215f9e5ca08227a9b'. This may indicate a malformed checkpoint.

I reproduced this with the following Qwen2.5 models listed as supported in the TensorRT Edge-LLM 0.10.0 Supported Models documentation:

  • LLM
    • Qwen2.5-3B -> ❌ export
    • Qwen2.5-3B-Instruct -> ❌ export
    • Qwen2.5-7B -> ❌ export
    • Qwen2.5-7B-Instruct -> ❌ export
  • VLM
    • Qwen2.5-VL-3B-Instruct -> ❌ export
    • Qwen2.5-VL-7B-Instruct -> ❌ export
    • Qwen2.5-VL-7B-Instruct-FP8 -> ❌ export
    • Qwen2.5-VL-7B-Instruct-NVFP4 -> ❌ export

Steps/Code to reproduce bug

Installation method:
As described in TensorRT Edge-LLM on Jetson, I used an NVIDIA Jetson Thor as both the host and target device. I installed TensorRT Edge-LLM v0.10.0 by following the Installation Guide. The installation completed successfully without any issues.

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jul_16_07:31:19_PM_PDT_2025
Cuda compilation tools, release 13.0, V13.0.48
Build cuda_13.0.r13.0/compiler.36260728_0

$ nvidia-smi
Mon Jul 27 08:52:49 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.00                 Driver Version: 580.00         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA Thor                    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   36C  N/A               3W /  N/A  | Not Supported          |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A           15507      G   /usr/lib/xorg/Xorg                       92MiB |
|    0   N/A  N/A           15707      G   /usr/bin/gnome-shell                     55MiB |
+-----------------------------------------------------------------------------------------+

$ python3 --version
Python 3.12.3

$ cat /etc/nv_tegra_release
# R38 (release), REVISION: 4.0, GCID: 43443517, BOARD: generic, EABI: aarch64, DATE: Wed Dec 31 00:15:19 UTC 2025
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
INSTALL_TYPE=

Export command used:
I use the following command to export models:

tensorrt-edgellm-export path/to/model_name model_name/onnx

For instance:

tensorrt-edgellm-export Qwen/Qwen2.5-VL-3B-Instruct Qwen2.5-VL-3B-Instruct/onnx
Details 11:59:59 INFO tensorrt_edgellm.scripts.export: Downloading Qwen/Qwen2.5-VL-3B-Instruct from Hugging Face Hub ... 11:59:59 INFO httpx: HTTP Request: GET https://huggingface.co/api/models/Qwen/Qwen2.5-VL-3B-Instruct/revision/main "HTTP/1.1 200 OK"

Fetching 14 files: 0%| | 0/14 [00:00<?, ?it/s]
Fetching 14 files: 100%|██████████| 14/14 [00:00<00:00, 1308.70it/s]
11:59:59 INFO tensorrt_edgellm.scripts.export: ============================================================
11:59:59 INFO tensorrt_edgellm.scripts.export: Model type : qwen2_5_vl
11:59:59 INFO tensorrt_edgellm.scripts.export: Checkpoint : /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct/snapshots/66285546d2b821cf421d4f5eb2576359d3770cd3
11:59:59 INFO tensorrt_edgellm.scripts.export: Output dir : Qwen2.5-VL-3B-Instruct/onnx
11:59:59 INFO tensorrt_edgellm.scripts.export: thinker : yes
11:59:59 INFO tensorrt_edgellm.scripts.export: mtp_draft : no
11:59:59 INFO tensorrt_edgellm.scripts.export: mtp_draft : no
11:59:59 INFO tensorrt_edgellm.scripts.export: dflash_draft : no
11:59:59 INFO tensorrt_edgellm.scripts.export: dspark_draft : no
11:59:59 INFO tensorrt_edgellm.scripts.export: talker : no
11:59:59 INFO tensorrt_edgellm.scripts.export: code_predictor : no
11:59:59 INFO tensorrt_edgellm.scripts.export: visual : yes
11:59:59 INFO tensorrt_edgellm.scripts.export: audio : no
11:59:59 INFO tensorrt_edgellm.scripts.export: rnnt_decoder : no
11:59:59 INFO tensorrt_edgellm.scripts.export: code2wav : no
11:59:59 INFO tensorrt_edgellm.scripts.export: action : no
11:59:59 INFO tensorrt_edgellm.scripts.export: FP8 embedding : no
11:59:59 INFO tensorrt_edgellm.scripts.export: MTP capable : no
11:59:59 INFO tensorrt_edgellm.scripts.export: MTP export : no
11:59:59 INFO tensorrt_edgellm.scripts.export: Gemma4 MTP : no
11:59:59 INFO tensorrt_edgellm.scripts.export: DFlash base : no
11:59:59 INFO tensorrt_edgellm.scripts.export: DFlash draft : no
11:59:59 INFO tensorrt_edgellm.scripts.export: DSpark base : no
11:59:59 INFO tensorrt_edgellm.scripts.export: DSpark draft : no
11:59:59 INFO tensorrt_edgellm.scripts.export: Reduced vocab : no
11:59:59 INFO tensorrt_edgellm.scripts.export: External weights: no
11:59:59 INFO tensorrt_edgellm.scripts.export: TP size : 1
11:59:59 INFO tensorrt_edgellm.scripts.export: ============================================================
11:59:59 INFO tensorrt_edgellm.scripts.export: [LLM] Loading checkpoint from /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct/snapshots/66285546d2b821cf421d4f5eb2576359d3770cd3
12:00:03 ERROR tensorrt_edgellm.scripts.export: [LLM] Failed to load checkpoint
Traceback (most recent call last):
File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 286, in _resolve_shard
resolved.relative_to(base)
File "/usr/lib/python3.12/pathlib.py", line 682, in relative_to
raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}")
ValueError: '/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct/blobs/41a8895c164b4d32bae6b302f4603fcbc1797f32dafa45c7e9bcda23c6755df8' is not in the subpath of '/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct/snapshots/66285546d2b821cf421d4f5eb2576359d3770cd3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/scripts/export.py", line 1030, in _export_llm
model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/model.py", line 543, in from_pretrained
load_weights(model,
File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 127, in load_weights
shard_map = _build_shard_map(model_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 337, in _build_shard_map
if not os.path.exists(_resolve_shard(model_dir, shard))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT-Edge-LLM/tensorrt_edgellm/checkpoint/loader.py", line 288, in _resolve_shard
raise ValueError(
ValueError: Shard path 'model-00001-of-00002.safetensors' in checkpoint index escapes model_dir '/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-VL-3B-Instruct/snapshots/66285546d2b821cf421d4f5eb2576359d3770cd3'. This may indicate a malformed checkpoint.

System information (Nvidia Jetson Thor)

  • Container used: nvcr.io/nvidia/pytorch:25.12-py3
  • OS: Ubuntu 24.04.3 LTS
  • CPU architecture: aarch64
  • GPU name: Nvidia Jetson T5000
  • GPU memory size: 128 GB
  • Number of GPUs: 1
  • Library versions:
    • Python: 3.12.3
    • TensorRT Edge-LLM version: 0.10.0
    • CUDA: 13.1
    • PyTorch: 2.13.0+cu130
    • Transformers: 5.14.1
    • ModelOpt: 0.45.0
    • ONNX: 1.19.0

Root Cause

The failure appears to be caused by the _resolve_shard implementation introduced in TensorRT Edge-LLM v0.10.0.

Previously, the shard was checked with: os.path.exists(os.path.join(model_dir, shard)).

In v0.10.0, _resolve_shard calls Path.resolve() before checking that the resulting path is inside model_dir.

This is incompatible with the standard Hugging Face Hub cache layout.

For example, the Qwen2.5-VL-3B-Instruct snapshot contains:

snapshots/
└── 66285546d2b821cf421d4f5eb2576359d3770cd3/
    └── model-00002-of-00002.safetensors

where the shard is a symlink:

model-00002-of-00002.safetensors -> ../../blobs/365531ff8752420e89dee707b79d021fb2d6e25abafe486f080555a4fe6972e4

Therefore: pathlib.Path(model_dir).resolve() points to the snapshots/ directory, while: (base / shard).resolve() follows the symlink and points to the corresponding file in blobs/.

Consequently: resolved.relative_to(base) raises ValueError, even though the shard is a valid file belonging to the Hugging Face model snapshot.

This is the relevant cache structure:

models--Qwen--Qwen2.5-VL-3B-Instruct/
├── blobs/
│     └── 365531ff8752420e89dee707b79d021fb2d6e25abafe486f080555a4fe6972e4
└── snapshots/
      └── 66285546d2b821cf421d4f5eb2576359d3770cd3/
            └── model-00002-of-00002.safetensors -> ../../blobs/365531ff8752420e89dee707b79d021fb2d6e25abafe486f080555a4fe6972e4

Possible fix

The containment check can be performed on the path without resolving the shard symlink:

def _resolve_shard(model_dir: str, shard: str) -> str:
    """Return the absolute shard path, asserting it stays inside model_dir."""
    base = pathlib.Path(model_dir).resolve()
    candidate = base / shard

    # Don't allow absolute shard path
    if pathlib.Path(shard).is_absolute():
        raise ValueError(f"Absolute shard path is not allowed: {shard!r}")

    # Validate path without following symlinks
    candidate_abs = candidate.absolute()
    try:
        candidate_abs.relative_to(base)
    except ValueError:
        raise ValueError(
            f"Shard path {shard!r} escapes model_dir {model_dir!r}."
        )

    if not candidate_abs.exists():
        raise FileNotFoundError(candidate_abs)

    return str(candidate_abs)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions