docker环境配置--load-4bit报错No package metadata was found for bitsandbytes

[madaoxjw]

运行脚本：python sample_t2i.py --prompt "渔舟唱晚" --load-4bit

环境为官方给的docker环境：
sys.platform: linux
Python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
CUDA available: True
MUSA available: False
numpy_random_seed: 2147483648
GPU 0: NVIDIA A10
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 12.4, V12.4.99
GCC: x86_64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
PyTorch: 2.3.0a0+40ec155e58.nv24.03
PyTorch compiling details: PyTorch built with:

• GCC 11.2
• C++ Version: 201703
• Intel(R) oneAPI Math Kernel Library Version 2021.1-Product Build 20201104 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v3.3.2 (Git Hash N/A)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 12.4
• NVCC architecture flags: -gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_72,code=sm_72;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_87,code=sm_87;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_90,code=compute_90
• CuDNN 90.0 (built against CUDA 12.3)
• Magma 2.6.2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.4, CUDNN_VERSION=9.0.0, CXX_COMPILER=/opt/rh/gcc-toolset-11/root/usr/bin/c++, CXX_FLAGS=-fno-gnu-unique -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

TorchVision: 0.18.0a0
OpenCV: 4.7.0

错误信息：
Traceback (most recent call last):
File "/workspace/sample_t2i.py", line 31, in
args, gen, enhancer = inferencer()
File "/workspace/sample_t2i.py", line 22, in inferencer
enhancer = DialogGen(str(models_root_path / "dialoggen"), args.load_4bit)
File "/workspace/mllm/dialoggen_demo.py", line 164, in init
self.models = init_dialoggen_model(model_path, load_4bit=load_4bit)
File "/workspace/mllm/dialoggen_demo.py", line 56, in init_dialoggen_model
tokenizer, model, image_processor, context_len = load_pretrained_model(
File "/workspace/mllm/llava/model/builder.py", line 54, in load_pretrained_model
kwargs["quantization_config"] = BitsAndBytesConfig(
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/quantization_config.py", line 281, in init
self.post_init()
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/quantization_config.py", line 327, in post_init
if self.load_in_4bit and not version.parse(importlib.metadata.version("bitsandbytes")) >= version.parse(
File "/usr/lib/python3.10/importlib/metadata/init.py", line 996, in version
return distribution(distribution_name).version
File "/usr/lib/python3.10/importlib/metadata/init.py", line 969, in distribution
return Distribution.from_name(distribution_name)
File "/usr/lib/python3.10/importlib/metadata/init.py", line 548, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes

[madaoxjw]

问题解决简单记录

解决步骤

1. 克隆仓库并进入目录

   git clone https://gitcode.com/gh_mirrors/bi/bitsandbytes.git && cd bitsandbytes/

2. 编译配置

   cmake -DCOMPUTE_BACKEND=cuda -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc -S .

3. 执行构建

   make

4. 修改源代码文件

   编辑 /workspace/bitsandbytes/bitsandbytes/autograd/_functions.py 文件，找到大约第51行的代码：

   原代码：

   _is_compiling = torch.compiler.is_compiling

   修改后代码：

   try:
       _is_compiling = torch.compiler.is_compiling
   except AttributeError:
       def _is_compiling():
           return False

5. 安装包

   pip install . -i https://pypi.tuna.tsinghua.edu.cn/simple --no-deps --force-reinstall

参考资料

• PyTorch模型转换及部署实战指南