We have published paper on Arxiv for this paper! Paper
The DigiPsych Voice Analysis Pipeline was created to enable easy extraction of features for voice computing technologies. From the features extracted, users can model affective mood, neurodegenerative diseases, among a wide array of applications (ie: diagnosing colds, presence of caffeine, medication, etc.). The DigiPsych Voice Analysis Pipeline is the amalgamation of work performed by students in the UW DigiPsych Lab (http://www.rezahosseinighomi.com/)
The DigiPsych Voice Analysis pipeline also contains an in-house analysis pipeline that users can use to quickly model their data (https://github.com/larryzhang95/Voice-Analysis-Pipeline/tree/master/DigiPsych_API/Data_Science_API).
In order to utilize the Voice Analysis Pipeline, please make sure you are in a Python3 Environment.
Please make sure to have the following dependencies installed:
- nltk
- spacy
- numpy
- pandas
- textblob
- mlxtend
- sklearn
- seaborn
- matplotlib
For additional information, please refer to the wiki (https://github.com/larryzhang95/Voice-Analysis-Pipeline/wiki)
Gemaps Features are explained at the following link: https://sail.usc.edu/publications/files/eyben-preprinttaffc-2015.pdf
Avec Features are explained at the following link: https://ibug.doc.ic.ac.uk/media/uploads/documents/avec2013.pdf
NLTK Features and implementation details explained on their documentation link: https://www.nltk.org/api/nltk.html
Spacy Features and implementation details explained on their documentation link: https://spacy.io/api/
Please Make sure you are executing the pipeline in a Python3 Environment and have the following modules/packages installed:
pip install pandas scikit-learn textblob librosa nltk spacy librosa seaborn matplotlib speech_recognition
python featurize.py -a <Enter Audio Folder> -l <gemaps> <avec> <librosa> <all> #Extracting audio
python featurize.py -t <Enter Transcript Folder> -l <nltk> <spacy> <all> #Extracting transcript
python Voice_Feature_Wrapper.py
- Provides AVEC2013 Features
- Provides GeMAPS Features
- Provides Librosa Features
python Language_Feature_Wrapper.py
- Provides SPACY Features
- Provides NLTK Features
- Provides Linguistic Features of Complexity
- Provides Semantic Coherence Features
- audEERING for OpenSmile Capability: https://www.audeering.com/
- Neurolex VoiceBook: https://www.neurolex.ai/voicebook/
- Semantic Coherence:
We would like to thank Neurolex Diagnostics for assisting in developing the basis for this work.
We also have an associated paper with this pipeline submitted to Interspeech 2019, and plan to continue adding features. If you would like to request a feature to be added, please file an issue, and corresponding papers/sources which our collaborators and students may be able to implement