Video-Based Human Emotion Recognition System Using Deep Learning Techniques

Abstract

This thesis explores deep learning models for emotion recognition in videos, suitable for systems with limited memory like robots and embedded-systems. The main contribution of this thesis is proposing two low-memory models, xception+LSTM and mini-xception+C3D, for video classification. Despite the fact that both models had a small number of parameters (<100k), they got competitive classification accuracies compared to much larger models with millions of parameters. Thesis is divided to 6 chapters, along with a list of figures, list of tables, list of abbreviations, list of symbols, and a bibliography.

• Chapter 1 gives a quick overview of why emotion recognition is useful in real world scenarios. • Chapter 2 gives a literature review on the topic of emotion recognition. I cover the history of emotion recognition systems before and after the deep-learning era, with more focus on deep-learning architectures used to classify both still images and videos. I also cover the most commonly used datasets for bench-marking different algorithms on the emotion recognition task. • Chapter 3 focuses on the theoretical background that my work is based upon. I give a quick overview of neural networks, their optimization algorithms, and some advanced NN architectures. I also cover some theoretical background related to the class imbalance problem and how to design models and evaluation criteria to deal with this problem. • Chapter 4 describes my own work in detail. It presents my proposed methods and models to tackle the emotion recognition in videos task.