"Learning Linguistic Structure by Combining Simple Models"   Young Stars Program

03/31/06: 10:30 AM, webcast
11th Floor Large Conference Room
Host: Patrick Pantel and Jafar Adibi, schedule

Abstract: There is only one complete language processing device to date: the human brain. Though there is debate on how much built-in bias human learners might have, we definitely acquire language in a primarily unsupervised fashion. In contrast, most computational approaches to language processing are based on supervised methods, which require hand-labeled corpora for training. Since most high-level NLP tasks, such as machine translation and question-answering, lack richly annotated corpora, unsupervised methods are extremely appealing -- provided they can be made to work. Unsupervised learning of linguistic structure presents a challenge which is common to all unsupervised systems: how to get the system to learn the right thing, and what to do if it does not. One success story for unsupervised methods is in the task of learning word alignments between two languages for machine translation. For this task, it has been shown that increasingly sophisticated models can learn increasingly correct structure, albeit at a high cost of engineering and conceptual complexity. In this talk, I will discuss an alternative to the design of complex models: the intersection of multiple simple ones. I'll first describe a new approach to word alignment. Our system combines two simple hidden Markov models (HMMs), one for each translation direction. The models are trained so that they both explain the observed data and also predict compatible hidden structures. The models' interaction biases the learning in a way which is difficult to achieve using a single model. Broadly, a single HMM imposes a one-to-many alignment constraint, while the intersection of two HMMs encourages more strict one-to-one matchings. Next, I will describe a system for the unsupervised learning of syntactic parse trees. This system again combines two simple models, one designed to model constituent boundaries, and another designed to model the recursive relationships between phrases. The models impose constraints on each other, allowing their combination to learn better tree structures than either component. On both of these tasks, our systems produce state-of-the-art results, despite (or perhaps because of) their simplicity.

About Dan Klein: Dan Klein is an assistant professor of computer science at UC Berkeley (PhD Stanford, MS Oxford, BA Cornell). Professor Klein's research focuses on natural language processing, including unsupervised learning, statistical parsing, information extraction, and machine translation. His academic honors include a British Marshall Fellowship, an inaugural Microsoft New Faculty Fellowship, and best paper awards at the ACL and EMNLP conferences.

Last updated: Mon Jun 19 17:44:06 2006