Describe the bug
Faithfullness metric returns 1.0 even if retrieval_context is empty
To Reproduce
Steps to reproduce the behavior:
metric = FaithfulnessMetric(model="...")
test_case = LLMTestCase(
input="xxx",
expected_output="xxx",
actual_output="xxx",
retrieval_context=[""], # this is a list of empty string
)
metric.measure(test_case)
I believe the required parameters check is being bypassed here and this might apply for any required param of any metric using the same check function. I only tested with above code snippet as of now.
Expected behavior
I expect an error or a 0.0 score.
Describe the bug
Faithfullness metric returns
1.0even ifretrieval_contextis emptyTo Reproduce
Steps to reproduce the behavior:
I believe the required parameters check is being bypassed here and this might apply for any required param of any metric using the same check function. I only tested with above code snippet as of now.
Expected behavior
I expect an error or a
0.0score.