-
Notifications
You must be signed in to change notification settings - Fork 304
Open
Description
If you compute F1 in a case where a class is neither predicted nor present in the ground truth data, a warning refers to using zero_division:
import evaluate
f1 = evaluate.load("f1")
print(f1.compute(predictions=[0,0,0,0,0], references=[0,1,0,1,2], average=None, labels=[0,1,2,3]))Output:
{'f1': array([0.57142857, 0. , 0. , 0. ])}
[...\.venv\Lib\site-packages\sklearn\metrics\_classification.py:1731](file:///.../.venv/Lib/site-packages/sklearn/metrics/_classification.py:1731): UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no true nor predicted samples. Use `zero_division` parameter to control this behavior.
_warn_prf(average, modifier, f"{metric.capitalize()} is", result.shape[0])
But this argument is not present:
import evaluate, numpy as np
f1 = evaluate.load("f1")
print(f1.compute(predictions=[0,0,0,0,0], references=[0,1,0,1,2], average=None, labels=[0,1,2,3], zero_division=np.nan))Output:
TypeError Traceback (most recent call last)
Cell In[4], [line 3](vscode-notebook-cell:?execution_count=4&line=3)
1 import evaluate, numpy as np
2 f1 = evaluate.load("f1")
----> [3](vscode-notebook-cell:?execution_count=4&line=3) print(f1.compute(predictions=[0,0,0,0,0], references=[0,1,0,1,2], average=None, labels=[0,1,2,3], zero_division=np.nan))
File ...\.venv\Lib\site-packages\evaluate\module.py:467, in EvaluationModule.compute(self, predictions, references, **kwargs)
465 inputs = {input_name: self.data[input_name][:] for input_name in self._feature_names()}
466 with temp_seed(self.seed):
--> [467](file:///.../.venv/Lib/site-packages/evaluate/module.py:467) output = self._compute(**inputs, **compute_kwargs)
469 if self.buf_writer is not None:
470 self.buf_writer = None
TypeError: F1._compute() got an unexpected keyword argument 'zero_division'
Metadata
Metadata
Assignees
Labels
No labels