The new version of datasets doesn't support load_metric

[arjunaskykok]

In Chapter 11, Fine-Tuning Representation Models for Classification, we have this code:

from datasets import load_metric
import numpy as np

def compute_metric(eval_pred):
    """Calculate F1 score"""
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    
    load_f1 = load_metric("f1")
    f1 = load_f1.compute(predictions=predictions, references=labels)["f1"]
    return {"f1": f1}

We would get this error:

ImportError: cannot import name 'load_metric' from 'datasets'

The load_metric has been removed from datasets in version 3. The replacement is to use the evaluate library.

However, when I used the evaluate library, I got different numbers for the f1 metric from the book.