ZeroDivisionError Problem with score = re_0_10_rating(line) / normalize in f_context_relevance ?

[AmineMekki01]

Hello,

I am using Trulens for evaluating a Rag app.

`from trulens_eval.feedback.provider.openai import AzureOpenAI
from trulens_eval import Feedback, Select

provider = AzureOpenAI(deployment_name="gpt-4-32k")
f_context_relevance = (
Feedback(
provider.context_relevance_with_cot_reasons, name="Context Relevance"
)
.on(Select.RecordCalls.retrieve.args.query)
.on(Select.RecordCalls.retrieve.rets.collect())
.aggregate(np.mean)
)`

i am facing the following error which is weird :
Traceback (most recent call last): File "/opt/conda/envs/genai-model-eval/lib/python3.9/runpy.py", line 197, in _run_module_as_main    return _run_code(code, main_globals, None, File "/opt/conda/envs/genai-model-eval/lib/python3.9/runpy.py", line 87, in _run_code    exec(code, run_globals) File "/home/oneai/oneai-mns-genai-model_lib/misc/eval_run.py", line 17, in <module>    evaluation_results = run_evaluation_pipeline( File "/home/oneai/oneai-mns-genai-model_lib/genair/genai/msat/evaluation/evaluation_pipeline.py", line 108, in run_evaluation_pipeline    section_results = eval_obj.run_eval(use_ragas, use_trulens, True) File "/home/oneai/oneai-mns-genai-model_lib/genair/genai/msat/evaluation/evaluation_metrics.py", line 231, in run_eval    trulens_results = self.trulens_metrics() File "/home/oneai/oneai-mns-genai-model_lib/genair/genai/msat/evaluation/evaluation_metrics.py", line 158, in trulens_metrics    context_relevance = f_context_relevance( File "/opt/conda/envs/genai-model-eval/lib/python3.9/site-packages/trulens_eval/feedback/feedback.py", line 444, in __call__    return self.imp(*args, **kwargs) File "/opt/conda/envs/genai-model-eval/lib/python3.9/site-packages/trulens_eval/feedback/provider/base.py", line 392, in context_relevance_with_cot_reasons    return self.generate_score_and_reasons(system_prompt, user_prompt, temperature) File "/opt/conda/envs/genai-model-eval/lib/python3.9/site-packages/trulens_eval/feedback/provider/base.py", line 253, in generate_score_and_reasons    score = re_0_10_rating(line) / normalize ZeroDivisionError: float division by zero
looked into the repo and normalize is set to 10.0 by default. Why this error then ?

[sfc-gh-nvytla]

Hi @AmineMekki01,

Thank you for reporting this. This issue seems to be related to an outdated implementation, and is no longer relevant to the existing codebase. However, I did investigate this issue, and have a PR up to resolve other potential divide-by-zero errors: #2124.

For now, I will be marking this discussion as answered. For future issues/bugs you face, and potential features/improvements you would like to see in the library, please feel free to add a new item here. Thanks!