When using the langchain-chatGLM project to run, the model does not report an error and can be used normally, but when it is changed to FastChat, the following error appears. What is the reason for this?
Environment
python=3.8.16,torch=1.12.1,
T4
model
chatGLM-6B-int4
Error
AttributeError: module 'torch.cuda' has no attribute 'OutOfMemoryError'
When using the langchain-chatGLM project to run, the model does not report an error and can be used normally, but when it is changed to FastChat, the following error appears. What is the reason for this?
Environment
python=3.8.16,torch=1.12.1,
T4
model
chatGLM-6B-int4
Error
AttributeError: module 'torch.cuda' has no attribute 'OutOfMemoryError'