Initial Kindly Model performing poorly because of imbalanced data

[AyiteyDjaba]

The dataset attached to the datasets folder needs to be tidied up. Some messages are wrongly labeled as being offensive. Also, the dataset is imbalanced i.e. (more than 80% of the data is one class) and it could be causing the model to perform poorly.

The first step is to clean up the dataset and then use data augmentation to generate more data to balance out the dataset.

[nathanfletcher]

@AyiteyDjaba thanks for bringing this up. Could you show some of the output of your findings from your Notebook? We could use it as the initial baseline and rerun it as the data improves.

[nathanfletcher]

@lacabra could you please assign this to @AyiteyDjaba ? I would be a co-assignee

[lacabra]

@nathanfletcher Done. But for the record, you have the same admin privileges that I do, so you can assign issues all the same :)

[ewheeler]

Agree with @AyiteyDjaba that the quality of the label annotations is concerning

Here are a few that stood out to me as incorrectly labeled as offensive:

• "i will not leave you"
• "leave me alone"
• "sleepyhead"
• "wake up"
• "wtf"
• "I know! So weird."
• "No one"
• "yea right"
• "No"
• "yes it was"

Others have achieved good results with imbalanced data in similar offensive language classification fine-tuning tasks--and a broad survey didn't find any research datasets that were balanced (!) so could be difficult to obtain or create a balanced set, even with augmentation and/or synthetic data. That said, imbalanced training sets certainly can lead to overfitting and limit generalization, so would strongly encourage expanding the training set and ensuring that the minority class represents at least 20% of the training set labels.

It might also be helpful to include definition or reference to a definition of bullying, as compared to offensive language more broadly or hate speech. And if the goal is to automate moderation and ensure 'safe spaces' online, then perhaps a multiclass approach that can discriminate among various classes of offensive language might be worth exploring. A survey of similar research includes a nice conceptual basis and links to various models and datasets related to these various classes of offensive speech (and related non-English language research)

However my biggest concern is the extremely small size of the training data set used for fine-tuning. In most NLP fine-tuning research, a training set with less than 10k samples is considered small and can result in inconsistent training runs with poor results. While there is some promising research that can achieve more stable, consistent results with extremely small training sets- but note that these extremely small training sets for fine-tuning have ~1000 samples or more.

I think it is pretty unlikely that supervised fine-tuning with only ~100 low quality samples will result in a model that performs better than the unmodified twitter-roberta-base-offensive model. We would need to compare evaluation metrics across multiple model training runs to understand whether 'catastrophic forgetting' or too-many fine-tuning iterations are causing the kinds of instability often observed when fine-tuning with very small datasets. I would definitely recommend putting concerted effort towards improving the size and quality of the training dataset to have at least several thousand high-quality samples that have been vetted by a credible human annotation process.

In order to get a better sense of how well kindly performs, I would recommend comparing the performance of the kindly model with the twitter-roberta-base-offensive model and other similar offensive speech classifier models in terms of accuracy, precision, f1 macro, and perplexity metrics on a well-labeled test set. Unfortunately while there are some existing, well-labeled hate speech datasets that are increasingly used for such model benchmarking/comparison purposes, in a quick search I wasn't able to find any such 'gold-standard' labeled test dataset specifically focused on bullying (creation of such a dataset might be a valuable and welcome contribution for UNICEF to make to the NLP field!).