This repository contains the code for the dissertation project on emotion recognition. The experiments conducted as part of this research can be replicated using the main.py file, where various parameters can be adjusted for each experiment.
Before running the project, ensure that you have Python 3.12 installed on your machine. Follow these steps to set up your environment:
-
Create a virtual environment:
Open a terminal and run the following command to create a virtual environment:
python3.12 -m venv venv
-
Activate the virtual environment:
\venv\Scripts\activate
-
Install dependencies
pip install -r requirements.txt
Note: The code is set up for parallel execution with CUDA (if available). If your machine supports it, installing CUDA is recommended to train the model faster.
Due to data privacy concerns, the datasets (/deap & /amigos) folders are not included in this repo. To replicate the results, you can include them as described bellow:
└── deap/
└── data_preprocessed_python/
├── s01.dat
├── s02.dat
├── ...
└── s32.dat
└── amigos/
└── preprocessed/
├── Data_preprocessed_P01/
├── Data_preprocessed_P01.mat
├── Data_preprocessed_P02/
├── Data_preprocessed_P02.mat
├── ...
└── Data_preprocessed_P40/
└── Data_preprocessed_P40.mat
-
Running for the first time: When running the project for the first time, ensure that the configuration variable
generate_imagesinmain.pyis set toTrue. This will generate the necessary PSD images for the project. -
Execute: To execute the code, simply run the
main.pyfile:
python main.py- Configuring the parameters to replicate the results: The value of each variable in will determine which experiment can be replicated. Below there is a table with the configuration for each experiment. The parameters columns are indicated with (P):
| Model | Validation type | Classes | Dataset | (P) model_EEGNet | (P) deap | (P) amigos | (P) cross_subject | (P) original_classes | (P) generate_labels | (P) valence | (P) arousal | (P) num_epochs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Modified EEGNet | Cross-subject | Valence | DEAP | True | True | False | True | True | False | True | False | 500 |
| Modified EEGNet | Cross-subject | Arousal | DEAP | True | True | False | True | True | False | False | True | 500 |
| Modified EEGNet | Cross-subject | Valence/Arousal | DEAP | True | True | False | True | True | False | True | True | 500 |
| Modified EEGNet | Within-subject | Valence | DEAP | True | True | False | False | True | False | True | False | 100 |
| Modified EEGNet | Within-subject | Arousal | DEAP | True | True | False | False | True | False | False | True | 100 |
| Modified EEGNet | Within-subject | Valence/Arousal | DEAP | True | True | False | False | True | False | True | True | 100 |
| Modified EEGNet | Cross-subject | Valence | AMIGOS | True | False | True | True | True | False | True | False | 200 |
| Modified EEGNet | Cross-subject | Arousal | AMIGOS | True | False | True | True | True | False | False | True | 200 |
| Modified EEGNet | Cross-subject | Valence/Arousal | AMIGOS | True | False | True | True | True | False | True | True | 200 |
| Modified EEGNet | Within-subject | Valence | AMIGOS | True | False | True | False | True | False | True | False | 200 |
| Modified EEGNet | Within-subject | Arousal | AMIGOS | True | False | True | False | True | False | False | True | 200 |
| Modified EEGNet | Within-subject | Valence/Arousal | AMIGOS | True | False | True | False | True | False | True | True | 200 |
| Modified EEGNet | Cross-subject | Valence | DEAP & AMIGOS | True | True | True | True | True | False | True | False | 100 |
| Modified EEGNet | Cross-subject | Arousal | DEAP & AMIGOS | True | True | True | True | True | False | False | True | 100 |
| Modified EEGNet | Cross-subject | Valence/Arousal | DEAP & AMIGOS | True | True | True | True | True | False | True | True | 100 |
| Modified EEGNet | Within-subject | Valence | DEAP & AMIGOS | True | True | True | False | True | False | True | False | 100 |
| Modified EEGNet | Within-subject | Arousal | DEAP & AMIGOS | True | True | True | False | True | False | False | True | 100 |
| Modified EEGNet | Within-subject | Valence/Arousal | DEAP & AMIGOS | True | True | True | False | True | False | True | True | 100 |
| EfficientNet v2 | Cross-subject | Angry/Happy/Sad/Pleasant/Neutral | DEAP | False | True | False | True | False | True | False | False | 30 |
| EfficientNet v2 | Cross-subject | Valence | DEAP | False | True | False | True | True | True | True | False | 30 |
| EfficientNet v2 | Cross-subject | Arousal | DEAP | False | True | False | True | True | True | False | True | 30 |
| EfficientNet v2 | Cross-subject | Valence/Arousal | DEAP | False | True | False | True | True | True | True | True | 30 |
model_EEGNet: Setting this toTruewill use the EEGNet model.Falsevalue will use the EfficientNet v2 modeldeap: Setting this toTruewill use the DEAP datasetamigos: Setting this toTruewill use the AMIGOS dataset. This can only be used for the EEGNet modelcross_subject: Setting this toTruewill use cross-subject classification.Falsevalue will use within-subject classificationscheduler: Setting this toTruewill use the reduction of learning rate on plateau. This can only be used for the EEGNet modeloriginal_classes: This defines if the original classes should be used from the datasets (valence, arousal, dominance, liking). If set toFalsethe classes will be relabelled (angry, happy, sad, pleasant, neutral). This can only be used for the EfficientNet v2.generate_images: Setting this toTruewill generate the PSD plots based on the DEAP datasetgenerate_labels: Setting this toTruewill re-generate the labels.csv file which is needed for the different experiments of EfficientNet v2valence: Setting this toTruewill validate the valence classarousal: Setting this toTruewill validate the arousal classnum_epochs: Defines the number of epochs that the model will be trained for
dataset.py: Contains the dataset classes to read raw data from numpy arrays and combine it with the labels. It also includes functions for splitting the subject data into cross-subject or within-subject batches.dataset_img.py: Contains the dataset classes to read data from the PSD images folder and combine it with the labels from the .csv file.generate_images.py: Used to collect data from the DEAP dataset and generate PSD plots for each channel and trial per subject.generate_labels.py: Used to regenerate the labels.csv file that is utilized by the EfficientNet v2 classifier.load_data.py: Loads data from the DEAP dataset and converts it into a numpy array.load_data_amigos.py: Loads data from the AMIGOS dataset, formats it to match DEAP's structure, and converts it into a numpy array.main.py: The main file used to configure experiment parameters and execute the code.models.py: Contains the model classes.preprocessing.py: Contains functions used to preprocess data before training the models. Specifically, it includes functions to segment timesteps into chunks and binarize label values.project_torch.py: Contains the orchestration steps for the EEGNet model. For example: loading data --> creating data loaders --> initializing the model --> starting training.project_torch_img.py: Contains the orchestration steps for the EfficientNet v2 model.training.py: Includes the training, validation, and testing logic for each model.