Techn. Fakultät Website deprecated and outdated. Click here for the new site. FAU-Logo

Ronak Kosti Ph.D.

Researcher in the Computer Vision (CV) group at the Pattern Recognition Lab of the Friedrich-Alexander-Universität Erlangen-Nürnberg

PhD Thesis
Visual Scene Context in Emotion Perception

Psychological studies show that the context of a setting, in addition to facial expression and body language, lends important information that conditions our perception of people's emotions. However, context's processing in the case of automatic emotion recognition has not been explored in depth, partly due to the lack of sufficient data. In this thesis we present EMOTIC, a dataset of images of people in various natural scenarios annotated with their apparent emotion. The EMOTIC database combines two different types of emotion representation: (1) a set of 26 emotion categories, and (2) the continuous dimensions of valence, arousal and dominance. We also present a detailed statistical and algorithmic analysis of the dataset along with the annotators' agreement analysis. CNN models are trained using EMOTIC, combining a person's features with those of the setting (context). Our results not only show how the context of a setting contributes important information for automatically recognizing emotional states but also promote further research in this direction.


Thesis is available via open access: Opens external link in new window

Code and Data: Opens external link in new window