-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
264.3 static library kineto_LIBRARY-NOTFOUND not found.
264.3 Call Stack (most recent call first):
264.3 /usr/local/lib/python3.12/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:125 (append_torchlib_if_found)
264.3 CMakeLists.txt:481 (find_package)
264.3
264.3
264.3 -- Found Torch: /usr/local/lib/python3.12/dist-packages/torch/lib/libtorch.so
264.3 -- Removing Torch generated placeholder CUDA architecture flags: -gencode arch=compute_80,code=sm_80.
302.9 -- Found ucxx: /src/tensorrt_llm/cpp/build/ucxx/build/ucxx-config.cmake (found version "0.42.0")
302.9 -- CMAKE_SYSTEM_PROCESSOR: x86_64
302.9 -- Operating System: ubuntu, 24.04
303.0 -- Using MPI_C_INCLUDE_DIRS: /opt/hpcx/ompi/include;/opt/hpcx/ompi/include/openmpi;/opt/hpcx/ompi/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent/include
303.0 -- Using MPI_C_LIBRARIES: /opt/hpcx/ompi/lib/libmpi.so
303.0 CMake Error at tensorrt_llm/CMakeLists.txt:136 (message):
303.0 The internal_cutlass_kernels library is truncated or incomplete. This is
303.0 usually caused by using Git LFS (Large File Storage) incorrectly. Please
303.0 try running command git lfs install && git lfs pull.
303.0
303.0
303.0 -- Configuring incomplete, errors occurred!
303.0 -- Using virtual environment at: /src/tensorrt_llm/.venv-3.12 (Python 3.12)
303.0 -- Ensuring virtualenv version >=20.29.1,<22.0 is installed...
303.0 -- Creating virtual environment in /src/tensorrt_llm/.venv-3.12...
303.0 -- Installing requirements from /src/tensorrt_llm/requirements-dev.txt into /src/tensorrt_llm/.venv-3.12...
303.0 -- Found conan executable at: /src/tensorrt_llm/.venv-3.12/bin/conan
303.0 CMake Configure command:
303.0 cmake -DCMAKE_BUILD_TYPE="Release" -DBUILD_PYT="ON" -DBINDING_TYPE="nanobind" -DBUILD_DEEP_EP="ON" -DBUILD_DEEP_GEMM="ON" -DNVTX_DISABLE="ON" -DBUILD_MICRO_BENCHMARKS=OFF -DBUILD_WHEEL_TARGETS="tensorrt_llm;nvinfer_plugin_tensorrt_llm;th_common;bindings;deep_ep;deep_gemm;benchmarks;executorWorker" -DPython_EXECUTABLE=/src/tensorrt_llm/.venv-3.12/bin/python3 -DPython3_EXECUTABLE=/src/tensorrt_llm/.venv-3.12/bin/python3 "-DCMAKE_CUDA_ARCHITECTURES=all" -DTensorRT_ROOT=/usr/local/tensorrt -DCMAKE_TOOLCHAIN_FILE=/src/tensorrt_llm/cpp/build/conan/conan_toolchain.cmake -S "/src/tensorrt_llm/cpp"
303.0 Traceback (most recent call last):
303.0 File "/src/tensorrt_llm/scripts/build_wheel.py", line 1054, in
303.0 main(**vars(args))
303.0 File "/src/tensorrt_llm/scripts/build_wheel.py", line 637, in main
303.0 build_run(cmake_configure_command)
303.0 File "/usr/lib/python3.12/subprocess.py", line 571, in run
303.0 raise CalledProcessError(retcode, process.args,
303.0 subprocess.CalledProcessError: Command 'cmake -DCMAKE_BUILD_TYPE="Release" -DBUILD_PYT="ON" -DBINDING_TYPE="nanobind" -DBUILD_DEEP_EP="ON" -DBUILD_DEEP_GEMM="ON" -DNVTX_DISABLE="ON" -DBUILD_MICRO_BENCHMARKS=OFF -DBUILD_WHEEL_TARGETS="tensorrt_llm;nvinfer_plugin_tensorrt_llm;th_common;bindings;deep_ep;deep_gemm;benchmarks;executorWorker" -DPython_EXECUTABLE=/src/tensorrt_llm/.venv-3.12/bin/python3 -DPython3_EXECUTABLE=/src/tensorrt_llm/.venv-3.12/bin/python3 "-DCMAKE_CUDA_ARCHITECTURES=all" -DTensorRT_ROOT=/usr/local/tensorrt -DCMAKE_TOOLCHAIN_FILE=/src/tensorrt_llm/cpp/build/conan/conan_toolchain.cmake -S "/src/tensorrt_llm/cpp"' returned non-zero exit status 1.
Dockerfile.multi:134
133 | ARG BUILD_WHEEL_SCRIPT="scripts/build_wheel.py"
134 | >>> RUN --mount=type=cache,target=/root/.cache/pip --mount=type=cache,target=${CCACHE_DIR}
135 | >>> GITHUB_MIRROR=$GITHUB_MIRROR python3 ${BUILD_WHEEL_SCRIPT} ${BUILD_WHEEL_ARGS}
136 |
ERROR: failed to build: failed to solve: process "/bin/bash -c GITHUB_MIRROR=$GITHUB_MIRROR python3 ${BUILD_WHEEL_SCRIPT} ${BUILD_WHEEL_ARGS}" did not complete successfully: exit code: 1
make: *** [Makefile:81: release_build] Error 1
make: Leaving directory '/home/noname/projects/TensorRT-LLM/docker'
Run the command as prompted by the error message:
➜ TensorRT-LLM git:(main) git lfs pull
Username for 'https://github.com/': , 0 B | 0 B/s
It prompts for the GitHub.com username and password. After entering them multiple times, it still keeps prompting for the username and password. After investigation, quite a number of people online have also encountered this issue.
Metadata
Metadata
Assignees
Labels
No labels