It seems that torchcodec is not compatible with the new v25 release because they don't have cu129 wheels, only cu126. Likely related to #48395 . It probably works on cu13 because torchcodec >= 0.14 has cu13 wheels, but
Traceback (most recent call last):
vllm", line 10, in <module>
sys.exit(main())
^^^^^^
vllm/vllm/entrypoints/cli/main.py", line 18, in main
import vllm.entrypoints.cli.benchmark.main
vllm/vllm/entrypoints/cli/benchmark/main.py", line 10, in <module>
from vllm.entrypoints.serve.utils.api_utils import VLLM_SUBCMD_PARSER_EPILOG
vllm/vllm/entrypoints/serve/utils/api_utils.py", line 20, in <module>
from vllm.engine.arg_utils import EngineArgs
vllm/vllm/engine/arg_utils.py", line 119, in <module>
from vllm.v1.sample.logits_processor import LogitsProcessor
vllm/vllm/v1/sample/logits_processor/__init__.py", line 15, in <module>
from vllm.sampling_params import SamplingParams
vllm/vllm/sampling_params.py", line 23, in <module>
from vllm.v1.serial_utils import PydanticMsgspecMixin
vllm/vllm/v1/serial_utils.py", line 25, in <module>
from vllm.multimodal.inputs import (
vllm/vllm/multimodal/__init__.py", line 3, in <module>
from .hasher import MultiModalHasher
vllm/vllm/multimodal/hasher.py", line 17, in <module>
from .media import MediaWithBytes
vllm/vllm/multimodal/media/__init__.py", line 5, in <module>
from .connector import MEDIA_CONNECTOR_REGISTRY, MediaConnector
vllm/vllm/multimodal/media/connector.py", line 28, in <module>
from vllm.multimodal.video import get_video_loader_backend_for_processor
vllm/vllm/multimodal/video.py", line 36, in <module>
from torchcodec.decoders import VideoDecoder
vllm/.venv/lib/python3.12/site-packages/torchcodec/__init__.py", line 12, in <module>
from . import decoders, encoders, samplers, transforms # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
vllm/.venv/lib/python3.12/site-packages/torchcodec/decoders/__init__.py", line 7, in <module>
from .._core import AudioStreamMetadata, VideoStreamMetadata
vllm/.venv/lib/python3.12/site-packages/torchcodec/_core/__init__.py", line 8, in <module>
from ._metadata import (
vllm/.venv/lib/python3.12/site-packages/torchcodec/_core/_metadata.py", line 15, in <module>
from torchcodec._core.ops import (
vllm/.venv/lib/python3.12/site-packages/torchcodec/_core/ops.py", line 38, in <module>
load_torchcodec_shared_libraries()
vllm/.venv/lib/python3.12/site-packages/torchcodec/_internally_replaced_utils.py", line 111, in load_torchcodec_shared_libraries
raise RuntimeError(
RuntimeError: Could not load libtorchcodec. Likely causes:
1. FFmpeg is not properly installed in your environment. We support
versions 4, 5, 6, 7, and 8, and we attempt to load libtorchcodec
for each of those versions. Errors for versions not installed on
your system are expected; only the error for your installed FFmpeg
version is relevant. On Windows, ensure you've installed the
"full-shared" version which ships DLLs.
2. The PyTorch version (2.11.0+cu129) is not compatible with
this version of TorchCodec. Refer to the version compatibility
table:
https://github.com/pytorch/torchcodec?tab=readme-ov-file#installing-torchcodec.
3. Another runtime dependency; see exceptions below.
The following exceptions were raised as we tried to load libtorchcodec:
[start of libtorchcodec loading traceback]
FFmpeg version 8:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1503, in load_library
ctypes.CDLL(path)
File ".local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/ctypes/__init__.py", line 379, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libnvrtc.so.13: cannot open shared object file: No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torchcodec/_internally_replaced_utils.py", line 93, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1505, in load_library
raise OSError(f"Could not load this library: {path}") from e
OSError: Could not load this library: vllm/.venv/lib/python3.12/site-packages/torchcodec/libtorchcodec_core8.so
FFmpeg version 7:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1503, in load_library
ctypes.CDLL(path)
File ".local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/ctypes/__init__.py", line 379, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libnvrtc.so.13: cannot open shared object file: No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torchcodec/_internally_replaced_utils.py", line 93, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1505, in load_library
raise OSError(f"Could not load this library: {path}") from e
OSError: Could not load this library: vllm/.venv/lib/python3.12/site-packages/torchcodec/libtorchcodec_core7.so
FFmpeg version 6:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1503, in load_library
ctypes.CDLL(path)
File ".local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/ctypes/__init__.py", line 379, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libnvrtc.so.13: cannot open shared object file: No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torchcodec/_internally_replaced_utils.py", line 93, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1505, in load_library
raise OSError(f"Could not load this library: {path}") from e
OSError: Could not load this library: vllm/.venv/lib/python3.12/site-packages/torchcodec/libtorchcodec_core6.so
FFmpeg version 5:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1503, in load_library
ctypes.CDLL(path)
File ".local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/ctypes/__init__.py", line 379, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libnvrtc.so.13: cannot open shared object file: No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torchcodec/_internally_replaced_utils.py", line 93, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1505, in load_library
raise OSError(f"Could not load this library: {path}") from e
OSError: Could not load this library: vllm/.venv/lib/python3.12/site-packages/torchcodec/libtorchcodec_core5.so
FFmpeg version 4:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1503, in load_library
ctypes.CDLL(path)
File ".local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/ctypes/__init__.py", line 379, in __init__
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libnvrtc.so.13: cannot open shared object file: No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
vllm/.venv/lib/python3.12/site-packages/torchcodec/_internally_replaced_utils.py", line 93, in load_torchcodec_shared_libraries
torch.ops.load_library(core_library_path)
vllm/.venv/lib/python3.12/site-packages/torch/_ops.py", line 1505, in load_library
raise OSError(f"Could not load this library: {path}") from e
OSError: Could not load this library: vllm/.venv/lib/python3.12/site-packages/torchcodec/libtorchcodec_core4.so
[end of libtorchcodec loading traceback].[project]
name = "vllmtest"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.11"
dependencies = [
"vllm==0.24.0",
"torch",
"torchvision",
"torchaudio",
"torchcodec>=0.14",
"transformers>=5.5.*",
]
[tool.uv.sources]
torch = [
{ index = "pytorch-cu129" },
]
torchaudio = [
{ index = "pytorch-cu129" },
]
torchcodec = [
{ index = "pytorch-cu126" },
]
torchvision = [
{ index = "pytorch-cu129" },
]
vllm = [
{ index = "vllm-cu129" },
]
[[tool.uv.index]]
name = "pytorch-cu129"
url = "https://download.pytorch.org/whl/cu129"
explicit = true
[[tool.uv.index]]
name = "pytorch-cu126"
url = "https://download.pytorch.org/whl/cu126"
explicit = true
[[tool.uv.index]]
name = "vllm-cu129"
url = "https://wheels.vllm.ai/0.24.0/cu129"
explicit = trueNote that torchcodec doesn't have a cu129 wheel. If I leave it unspecified, it falls back to the cpu wheel and the same error occurs
Your current environment
The output of
python collect_env.py🐛 Describe the bug
It seems that torchcodec is not compatible with the new v25 release because they don't have cu129 wheels, only cu126. Likely related to #48395 . It probably works on cu13 because torchcodec >= 0.14 has cu13 wheels, but
Note that torchcodec doesn't have a cu129 wheel. If I leave it unspecified, it falls back to the cpu wheel and the same error occurs
Before submitting a new issue...