Not getting Streaming response with LangchainLLM adapter.

[manishdighore]

Overview:
I'm encountering an issue with the LangChainLLMs adapter not delivering streaming text chunks from my Mistral-7B model, despite enabling streaming and setting up the adapter correctly. Here's my setup:

from langchain_community.llms.openai import OpenAI

# Initializing Mistral-7B model with streaming enabled
url = "http://localhost:8000/v1"
mistral = OpenAI(base_url=url, api_key="EMPTY", model="mistralai/Mistral-7B-Instruct-v0.2",
                 temperature=0, max_tokens=128, streaming=True)

'''

When I directly call .stream on Mistral-7B, I receive streaming text chunks. However, when I use LangChainLLMs:

from gptcache.adapter.langchain_models import LLM, LangChainLLMs

# Initializing LangChainLLMs adapter
llm = LangChainLLMs(llm=mistral, verbose=True)

Streaming text chunks are not received through the adapter, even without cache hits.

Expected Behavior:
I expect LangChainLLMs to deliver streaming text chunks from Mistral-7B, especially when there's no cache hit.

Actual Behavior:
Streaming text chunks are not received through LangChainLLMs, impacting real-time interaction with the model.

[SimFG]

if you want to use the gptcache with the langchain, i suggest you can try to use the set_llm_cache method in the langchain.
refer to: #585 (comment)