Abstract

To accomplish effective communication, interaction partners generally adapt their verbal and non-verbal behavior to that of their interlocutors. This behavior adaptation is often modulated by the underlying emotional states of partners. Modeling such mutual behavioral influence is critical for emotion characterization in an interaction. In this paper, we focus on explicitly modeling the mutual influence of multimodal behavior (speech and hand gesture) in affective dyadic interactions. In our framework, the behavior adaptation in each interaction is modeled by an interaction matrix which assembles all behavioral information on the path between the dyad's behavior. Experimental results show that our modeling approach can significantly improve the performance of emotion recognition. We further investigate the properties of the interaction model. Analysis results reveal that the entrainment effect of dyad's behavior can be …

Date

April 19, 2015

Authors

Zhaojun Yang, Shrikanth Narayanan

Conference

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Pages

2234-2238

Publisher

IEEE