Abstract

In this paper, a data-driven audio-based head motion synthesis technique is presented for avatar-based telepresence systems. First, head motion of a human subject speaking a custom corpus is captured, and the accompanying audio features are extracted. Based on the aligned pairs between audio features and head motion (audio-headmotion), a K-Nearest Neighbors (KNN) based dynamic programming algorithm is used to synthesize novel head motion given new audio input. This approach also provides optional intuitive keyframe (key head poses) control: after key head poses are specified, this method will synthesize appropriate head motion sequences that maximally meet the requirements of both the speech and key head poses.

Date

2004

Authors

Zhigang Deng, Shri Narayanan, Carlos Busso, Ulrich Neumann

Book

Proceedings of the 2004 ACM SIGMM workshop on Effective telepresence

Pages

24-30