Abstract

Emotion perception underlies communication and social interaction, shaping how we interpret our world. However, there are many aspects of this process that we still do not fully understand. Notably, we have not yet identified how audio and video information are integrated during the perception of emotion. In this work we present an approach to enhance our understanding of this process using the McGurk effect paradigm, a framework in which stimuli composed of mismatched audio and video cues are presented to human evaluators. Our stimuli set contain sentence-level emotional stimuli with either the same emotion on each channel (“matched”) or different emotions on each channel (“mismatched”, for example, an angry face with a happy voice). We obtain dimensional evaluations (valence and activation) of these emotionally consistent and noisy stimuli using crowd sourcing via Amazon Mechanical Turk. We …

Date

2013

Authors

Emily Mower Provost, Irene Zhu, Shrikanth Narayanan

Conference

2013 IEEE International Conference on Multimedia and Expo (ICME)

Pages

1-6

Publisher

IEEE