Event Details

I will present a couple of research vignettes of the working knowledge inside large pre-trained language models. I will use the vignettes to argue for a new task to measure modern language models’ knowledge and ability to learn from textbooks. Unlike machines, humans do not need to read, for example, all of Wikipedia, to learn. For humans, reading a textbook or a manual is often enough to provide working knowledge on the book’s topic. We propose LEFT, a new task to measure a machine’s capacity to learn from the same textbooks that college graduates use to learn about society and history. The task reveals surprising results for current state-of-the-art language models like T5 and GPTNeo.

Reminder: ​Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please inform [email protected] beforehand so we’ll be aware of your attendance and let you in.

Speaker Bio

Manuel Ciosici is a postdoc at ISI Boston (Waltham), working with Ralph Weischedel and Marjorie Friedman on understanding the knowledge inside large language models and putting it to use, for example, in filling in sequences of events. He is also interested in Natural Language Processing for languages other than English and has recently released a large corpus of Danish to support training large language models. Before joining ISI, Manuel received his Ph.D. from Aarhus University in Denmark and was a postdoc at the IT University in Copenhagen.