Hate speech on online platforms poses a significant societal challenge by spreading discrimination and harm against targeted communities. While neural models achieve strong classification performance, they often lack transparency and may reinforce hidden biases — a serious concern for automated content moderation applied to marginalised groups.
This project builds an explainable hate-speech detection system on HateXplain, the first hate-speech dataset that annotates not just class labels but also per-token rationale masks indicating which words each annotator used to justify their decision. We frame the problem as three-class classification (hatespeech / offensive / normal) and jointly train a classifier with a token-level rationale head, evaluating the resulting explanations along four axes:
- Plausibility — overlap between model rationales and human-annotated spans (Recall@K, Token-F1, IOU).
- Faithfulness — whether those tokens actually drive the prediction when masked (comprehensiveness, sufficiency, AOPC).
- Post-hoc attribution — LIME as a model-agnostic baseline on the same encoder.
- Human-centric fairness — persona-infused LLM annotations simulating in-group vs out-group annotator perspectives.
- Milestone 1 — Literature survey and approach selection
- Milestone 2 — First multi-task implementation and results
- Milestone 3 — Final results and cross-implementation comparison
HateSpeechExplain/
├── src/ # all source code (function-based, not author-based)
│ ├── data/ # HateXplain loader, expanded counterfactual probe
│ ├── models/ # RationaleSupervisedClassifier
│ ├── training/ # unified rationale trainer; Stage-1 trainers; LIME baseline
│ ├── evaluation/ # plausibility, faithfulness, counterfactual, LIME
│ ├── analysis/ # comparison table, persona PoC, faithfulness summary
│ ├── utils/ # device, model loading
│ └── demo.py # live demo entry point
├── data/ # HateXplain raw + Davidson dataset
├── models/ # pretrained encoders (weights gitignored)
├── outputs/ # produced artifacts
│ ├── checkpoints/ # trained model weights (gitignored)
│ ├── explanations/ # per-example JSONL exports
│ ├── faithfulness/ # per-checkpoint comp/suff/AOPC summaries
│ ├── counterfactuals/ # generated probe sets
│ ├── tables/ # comparison CSVs, per-model metrics
│ └── persona/ # LLM annotation outputs
├── docs/ # milestone reports (PDF)
├── notebooks/ # exploratory notebooks (legacy)
├── analysis/ # legacy attention-based exploration scripts
└── cluster/ # Singularity definition for HPC use
Conda env on Windows (PowerShell). For RTX 50-series (Blackwell) GPUs the standard PyTorch wheels don't include sm_120 — use the CUDA 12.8 build.
conda create -n hatespeech python=3.10 -y
conda activate hatespeech
# CUDA 12.8 (sm_120 / RTX 50-series). Fall back to /nightly/cu128 if stable
# lacks a wheel for your platform. Use /cu121 if your driver is older.
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
pip install ipykernel jupyterlab ipywidgets
python -m ipykernel install --user --name hatespeech --display-name "HateSpeechEnv"The live demo — predicts and shows rationales for a few examples plus a counterfactual probe:
python -m src.demoCross-implementation results on the HateXplain test set (1,924 examples). Dynamic-K columns use the per-example gold rationale count as K. SuffGap is lower-is-better; all other metrics are higher-is-better.
| Configuration | F1-macro | Recall@10 | TF1 (dyn-K) | Comp (dyn-K) | SuffGap (dyn-K) ↓ | AOPC (dyn-K) |
|---|---|---|---|---|---|---|
| BERT λ=0.0 | 0.676 | 0.521 | 0.316 | 0.141 | 0.392 | 0.090 |
| BERT λ=1.0 | 0.682 | 0.841 | 0.870 | 0.559 | 0.208 | 0.499 |
| BERT+Davidson λ=0.0 | 0.663 | 0.664 | 0.395 | 0.331 | 0.396 | 0.240 |
| BERT+Davidson λ=1.0 | 0.673 | 0.837 | 0.852 | 0.440 | 0.145 | 0.404 |
| RoBERTa λ=0.0 | 0.688 | 0.524 | 0.310 | 0.138 | 0.452 | 0.069 |
| RoBERTa λ=1.0 | 0.673 | 0.846 | 0.875 | 0.601 | 0.096 | 0.538 |
| LIME on BERT baseline | 0.649 | 0.724 | 0.680 | — | — | — |
| LIME on BERT+Davidson | 0.660 | 0.699 | 0.635 | — | — | — |
The core model is a RationaleSupervisedClassifier: a HuggingFace transformer encoder (BERT-base or RoBERTa-base) with two task heads sharing the same backbone:
- A classification head predicting one of {hatespeech, offensive, normal} from the pooled output.
- A rationale head predicting a binary score per token, indicating whether it belongs to the human rationale.
Both heads are trained jointly: L = L_CE + λ · L_BCE, where λ controls the weight of rationale supervision. Setting λ=0 reduces to a classification-only baseline; λ=1 is full supervision. A two-stage transfer learning pipeline was also evaluated: intermediate fine-tuning on the Davidson hate-speech dataset with gradual unfreezing, followed by fine-tuning on HateXplain.
Plausibility measures how much the model's top-K rationale tokens overlap with what human annotators highlighted. Metrics include Recall@K, Token-F1, and IOU at both fixed K=10 and dynamic K (matching the per-example human rationale length).
Rationale supervision raises Recall@10 from ~0.52 to ~0.84 across all three architectures, with no meaningful drop in classification accuracy. The result holds equally for BERT, BERT+Davidson, and RoBERTa.
Faithfulness measures whether the highlighted tokens actually drive the model's prediction, evaluated via token occlusion:
- Comprehensiveness: confidence drop when top-K tokens are masked. Higher means the model relied on those tokens.
- Sufficiency gap: confidence drop when only top-K tokens are kept. Lower means the rationale alone is enough to make the prediction.
- AOPC: area under the progressive-occlusion curve.
AOPC jumps from ~0.07–0.09 (unsupervised) to 0.40–0.54 (supervised) across architectures, confirming the model is genuinely using the highlighted tokens rather than just labelling them to look plausible.
LIME (LimeTextExplainer, 500 perturbations per example) was applied to 200 test examples on two encoders: a vanilla BERT baseline and the BERT+Davidson λ=0.0 checkpoint. The latter provides a direct apples-to-apples comparison against the supervised rationale head on the same encoder.
On the same encoder (BERT+Davidson), the three explanation sources rank cleanly on dynamic-K Token-F1:
- Unsupervised rationale head (λ=0.0): 0.40
- LIME post-hoc: 0.63
- Supervised rationale head (λ=1.0): 0.85
LIME beats the unsupervised rationale head, but training-time supervision wins by another 35% over LIME. When rationale annotations are available at training time, supervised rationales are the stronger option; when only a trained classifier is available, LIME is a real improvement over reading attention weights directly.
Inspired by Gajewska et al. (2025), this fairness experiment uses GPT-4.1-mini as a simulated annotator across three roles per example: a neutral baseline, an in-group persona (a member of the community targeted in the post), and an out-group persona. The balanced run covers 5 target groups (African, Islam, Jewish, Women, Homosexual) × 3 labels × 5 examples × 3 roles = 225 API calls.
| Role | Hate recall | FPR | 3-class accuracy |
|---|---|---|---|
| Neutral baseline | 0.840 | 0.280 | 0.613 |
| In-group persona | 1.000 | 0.340 | 0.680 |
| Out-group persona | 0.920 | 0.280 | 0.667 |
In-group personas missed zero hate examples at the cost of a 6pp higher false-positive rate, modestly reproducing the in-group sensitivity / specificity trade-off reported in the original paper.