Abstract

The study provides an empirical analysis of long-term user behavioral changes and varying user strategies during cross-lingual interaction using the multimodal speech-to-speech (S2S) translation system of USC/SAIL. The goal is to inform user adaptive designs of such systems. A 4-week medical-scenario-based study provides the basis for our analysis. The data analyzed includes user interviews, post-session surveys, and the extensive system logs that were post-processed and annotated. The annotations measured the meaning transfer rates using human evaluations and a scale defined here called the concept matching score. First, qualitative data analysis investigates user strategies in dealing with errors, such as repeat, rephrase, change topic, start over, and the participants’ self-reported longitudinal adaptation to errors. Post-session surveys explore participant experience with the system and point to a trend …

Date

February 1, 2013

Authors

JongHo Shin, Panayiotis G Georgiou, Shrikanth Narayanan

Journal

Computer Speech & Language

Volume

27

Issue

2

Pages

554-571

Publisher

Academic Press