SEC: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation

This repository presents a PyTorch implementation of meta-learning models applied on the DailyDialog dataset, performing emotions predictions. It consists in a Master thesis work for the Master of Data Science at the University of Luxembourg.

This work has been accepted at two venues:

• JEP-TALN 2024 "SEC : Contexte Émotionnel Phrastique pour la Reconnaissance Émotionnelle Efficiente dans la Conversation" (in French)
• WASSA @ ACL 2024 "SEC: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation"

Citations of both papers will soon be available in the Cite section.

Table of Contents

• Data
• Models
• Usage
  • Preprocessing
  • Training and evaluation
  • Qualitative insights
• Cite

Data

DailyDialog dataset was introduced by Li et al. in 2017 and consists in more than 12,000 generated dialogues intended to be representative of daily concerns. These dialogues are labelled with the 6 Ekman's emotions. More details about DailyDialog can be found here.

Models

In this work, we perform Emotion Recognition in Conversation (ERC) on DailyDialog conversation using meta-learning approaches. More precisely, this aim is to represent utterances from each dialog in its conversational context in order to predict accurate emotions. The model architecture used in the experiments in the Siamese Networks. Another architecture called MAML (Model-Agnostic Meta-Learning) is yet to be explored.

In the provided code for Siamese Networks, we implemented to models. The first one, referred to as isolated utterance model, is bound to perform emotion recognition on isolated utterances, without any contextual information at dialog level. The second one, referred to as contextual utterance model, uses a BERT-based encoding that takes the conversational context into account, as BERT tokenization is performed at dialog level. Below are illustrated the two different training pipeline of the aforementioned models:

Isolated utterance model

Contextual utterance model

Usage

The provided code allows to run data preprocessing, training and inference for both model. In order to achieve full reproducibility, please refer to the file requirements.txt by using:

pip install -r requirements.txt

To start with, clone the repository and go inside SiameseNetworks directory:

git clone https://github.com/B-Gendron/meta_dyda.git
cd meta_dyda
cd SiameseNetworks

It is possible to reproduce the following experiments:

• static utterance model training and evaluation
• contextual utterance model training and evaluation
• qualitative insights on static utterance model predictions
• qualitative insights on contextual utterance model predictions

Preprocessing

Because the processed datasets are not available in the repository, it is necessary to perform preprocessing before training on data. Three different preprocessing pipelines are available:

• Conversation-aware dialog representations
  This experiment is the main contribution of this software. It is required to perform preprocessing on both isolated and contextual data, because isolated utterance data will be use te train the auxiliary emotion classifier, while the contextual data is used to perform the main experiment, meaning emotion recognition in conversational context.

python3 preprocessing.py -l s -t u
python3 preprocessing.py -l s -t c

• Isolated utterance representations
  In this case that accounts for a baseline, of course only isolated data preprocessing is needed.

python3 preprocessing.py -l u

Training and evaluation

Please note that, due to their sizes, the models are not provided in this repository. Therefore, one needs to train the model in order to perform inference on it. Furthermore, the qualitative experiments can be performed only if the models are already stored.

To run the experiments, one calls main.py script with some arguments:

• -m for specifying the model (isolated or contextual)
• -t to run training and inference
• -q to run qualitative analysis The experiments triggered when specifying -t and -q depends on the model value in -m. This is therefore a required argument.

Below are given all commands for each experiment so they can be directly copied and pasted. If you encounter any issue running the following code snippets, don't hesitate to display help first:

python3 main.py -h

Use the following command to train the model you want. The isolated utterance model is based on static utterance representations, therefore it does not take the conversational context into account. The selected model in this case is a 5-layer LSTM. On the other hand, the contextual utterance model takes the conversational context into account. The selected model is based on Sentence BERT. Here is the command for the isolated utterance model:

python3 main.py -m i -t

And for contextual utterance model:

python3 main.py -m s -t

Qualitative insights

The following commands will proceed some qualitative analysis on parts of DailyDialog test set. Concretely, it will generate an output .txt file that contains dialogs along with the considered utterance and prediction compared to ground truth labels. For isolated utterance model:

python3 main.py -m i -q

And for contextual utterance model:

python3 main.py -m s -q

Cite the papers

JEP-TALN 2024

@inproceedings{Gendron-Guibon:JEP-TALN:2024,
    author = {Gendron, Barbara and Guibon, Ga\"el},
    title = "SEC : contexte \'emotionnel phrastique int\'egr\'e pour la reconnaissance \'emotionnelle efficiente dans la conversation",
    booktitle = "Actes de JEP-TALN-RECITAL 2024. 31\`eme Conf\'erence sur le Traitement Automatique des Langues Naturelles,  volume 1 : articles longs et prises de position",
    month = "7",
    year = "2024",
    address = "Toulouse, France",
    publisher = "Association pour le Traitement Automatique des Langues",
    pages = "219-233",
    note = "",
    abstract = "L'essor des mod\`eles d'apprentissage profond a apport\'e une contribution significative \`a la reconnaissance des \'emotions dans les conversations (ERC). Cependant, cette t\^ache reste un d\'efi important en raison de la pluralit\'e et de la subjectivit\'e des \'emotions humaines. Les travaux ant\'erieurs sur l'ERC fournissent des mod\`eles pr\'edictifs utilisant principalement des repr\'esentations de la conversation bas\'ees sur des graphes. Dans ce travail, nous proposons une fa\c{c}on de mod\'eliser le contexte conversationnel que nous incorporons \`a une strat\'egie d'apprentissage de m\'etrique, avec un processus en deux \'etapes. Cela permet d'effectuer l'ERC dans un sc\'enario de classification flexible et d'obtenir un mod\`ele l\'eger et efficace. En utilisant l'apprentissage de m\'etrique \`a travers une architecture de r\'eseau siamois, nous obtenons un score de macroF1 de 57,71\\% pour la classification des \'emotions dans les conversations sur le jeu de donn\'ees DailyDialog, ce qui surpasse les travaux connexes. Ce r\'esultat \'etat-de-l'art est prometteur en ce qui concerne l'utilisation de l'apprentissage de m\'etrique pour la reconnaissance des \'emotions, mais est perfectible au regard du microF1 obtenu.",
    keywords = "Apprentissage profond,Reconnaissance d'\'emotions en conversation,Apprentissage de m\'etrique",
    url = "https://talnarchives.atala.org/TALN/TALN-2024/7864.pdf"
}

WASSA 2024 (ACL workshop)

@inproceedings{gendron-guibon-2024-wassa,
    title = "{SEC}: Context-Aware Metric Learning for Efficient Emotion Recognition in Conversation",
    author = {Gendron, Barbara  and
      Guibon, Ga{\"e}l},
    editor = "De Clercq, Orph{\'e}e  and
      Barriere, Valentin  and
      Barnes, Jeremy  and
      Klinger, Roman  and
      Sedoc, Jo{\~a}o  and
      Tafreshi, Shabnam",
    booktitle = "Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, {\&} Social Media Analysis",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.wassa-1.2/",
    doi = "10.18653/v1/2024.wassa-1.2",
    pages = "11--22",
    abstract = "The advent of deep learning models has made a considerable contribution to the achievement of Emotion Recognition in Conversation (ERC). However, this task still remains an important challenge due to the plurality and subjectivity of human emotions. Previous work on ERC provides predictive models using mostly graph-based conversation representations. In this work, we propose a way to model the conversational context that we incorporate into a metric learning training strategy, with a two-step process. This allows us to perform ERC in a flexible classification scenario and end up with a lightweight yet efficient model. Using metric learning through a Siamese Network architecture, we achieve 57.71 in macro F1 score for emotion classification in conversation on DailyDialog dataset, which outperforms the related work. This state-of-the-art result is promising in terms of the use of metric learning for emotion recognition, yet perfectible compared to the micro F1 score obtained."
}