ragas evaluator function is taking LLM model as none even it is passed in evaluator's function call

[manaligc]

Code:
%python
# %pip install openai langchain

from ragas.metrics import answer_relevancy, faithfulness, context_utilization
from ragas.evaluation import evaluate
from azure.identity import DefaultAzureCredential
import openai

import os
from langchain.llms import AzureOpenAI

# Set your Azure OpenAI endpoint and API key
endpoint = <end point>
api_key = <api_key>
api_version = <api_version>

# Configure OpenAI client
os.environ["OPENAI_API_KEY"] = api_key

# Wrap the AzureOpenAI client with Langchain's AzureOpenAI class
evaluator_llm = AzureOpenAI(
    api_key=api_key,
    api_base=endpoint,
    api_type="azure",
    api_version=api_version,
    deployment=deployment
)

# Compute RAGAS Scores
scores = evaluate(
    dataset,
    metrics=[faithfulness, answer_relevancy, context_utilization],
    llm=evaluator_llm
)

Error:
AttributeError: 'AzureOpenAI' object has no attribute 'set_run_config'
File , line 25
16 evaluator_llm = AzureOpenAI(
17 api_key=api_key,
18 api_base=endpoint,
(...)
21 deployment=deployment
22 )
24 # Compute RAGAS Scores
---> 25 scores = evaluate(
26 dataset,
27 metrics=[faithfulness, answer_relevancy, context_utilization],
28 llm=evaluator_llm
29 )
31 # Convert results back to DataFrame for easy viewing
32 results_df = scores.to_pandas()
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-087f3445-139b-431e-884f-bffb273ec73f/lib/python3.11/site-packages/ragas/metrics/base.py:153, in MetricWithLLM.init(self, run_config)
149 if self.llm is None:
150 raise ValueError(
151 f"Metric '{self.name}' has no valid LLM provided (self.llm is None). Please initantiate a the metric with an LLM to run." # noqa
152 )
--> 153 self.llm.set_run_config(run_config)

[sahusiddharth]

Hi @manaligc,

Which ragas version are you using? If you are using the latest one.

I'd recommend importing the metrics class, Wrapping you Langchain LLM with ragas LangchainLLMWrapper

something like this

from ragas.metrics import AnswerAccuracy, ContextUtilization, Faithfulness
from langchain.llms import AzureOpenAI
from ragas.llms import LangchainLLMWrapper

evaluator_llm = LangchainLLMWrapper(AzureOpenAI(
    api_key=api_key,
    api_base=endpoint,
    api_type="azure",
    api_version=api_version,
    deployment=deployment
))

scores = evaluate(
    dataset,
    metrics=[AnswerAccuracy(llm=evaluator_llm), ContextUtilization(llm=evaluator_llm), Faithfulness(llm=evaluator_llm)],
    llm=evaluator_llm
)