Event Details

REMINDER: This talk will be Live Only, it will not be recorded.

Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom.

If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance and let you in.

In-person attendance will be permitted for USC/ISI faculty, staff, students only. Open to the public virtually via the zoom registration link and online.

For more information on the NL Seminar series and upcoming talks, please visit:

https://nlg.isi.edu/nl-seminar/ 

Abstract:

A crucial limitation of current sentence-level machine translation systems is their inability to account for context. By processing each sentence in isolation, existing neural machine translation NMT systems are prone to missing important document level cues and demonstrate a poor understanding of inter-sentential discourse properties, resulting in a noticeable quality difference between human translated and machine translated text. In this talk, we will discuss ongoing efforts to construct NMT models that can effectively harness context. We primarily focus on the popular IWSLT 17 English to French translation task, and compare against a strong concatenation based Transformer (Vaswani et al., 2017) baseline. First, we corroborate existing findings (Fernandes et al. 2021) that increasing context can improve translation performance, though with diminishing returns. We hypothesize that the Transformer’s self-attention mechanism may be insufficient for handling long range dependencies across sentences, both inside and outside of the context window. We then explore replacing the Transformer with a novel neural architecture whose attention layer is based on an exponential moving average to exploit both local and global contexts. Finally, we will discuss a chunk-based strategy towards encoding and decoding text, and conclude with future directions.

Speaker Bio

Jacqueline He is a current summer intern for the Natural Language Group at USC ISI under Professors Jonathan May and Xuezhe Ma. She recently graduated from Princeton University with a bachelor’s degree in Computer Science. Her current research interest orients around contextual aware neural machine translation, and she has previously worked on interpretability and ethics in NLP.