Seminars and Events
NL Seminar-DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models
Event Details
Speaker: Ollie Liu, USC
Conference Rm Location: ISI-MDR #689
REMINDER:
Meeting hosts only admit on-line guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom.
If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) to make us aware of your attendance so we can admit you. Specify if you will attend remotely or in person at least one business day prior to the event Provide your: full name, job title and professional affiliation and arrive at least 10 minutes before the seminar begins.
Abstract:
Large language models (LLMs) are increasingly used across society, including in domains like business, engineering, and medicine. These fields often grapple with decision-making under uncertainty, a critical yet challenging task. In this paper, we show that directly prompting LLMs on these types of decision-making problems yields poor results, especially as the problem complexity increases. To overcome this limitation, we propose DeLLMa (Decision-making Large Language Model assistant), a framework designed to enhance decision-making accuracy in uncertain environments. DeLLMa involves a multi-step scaffolding procedure, drawing upon principles from decision theory and utility theory, to provide an optimal and human-auditable decision-making process. We validate our framework on decision-making environments involving real agriculture and finance data. Our results show that DeLLMa can significantly improve LLM decision-making performance, achieving up to a 40% increase in accuracy over competing methods.
Speaker Bio
Ollie Liu (https://ollieliu.com/) is second-year Ph.D student in Computer Science at University of Southern California, co-advised by Prof. Dani Yogatama and Prof. Willie Neiswanger. In life, I usually go by Oliver 🫒 My current research interests lie in (multimodal) foundation models, especially their algorithmic reasoning capabilities and applications in sciences.
If speaker approves to be recorded for this NL Seminar talk, it will be posted on the USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI.
Subscribe here to learn more about upcoming seminars: https://www.isi.edu/events/
For more information on the NL Seminar series and upcoming talks, please visit:
https://www.isi.edu/research-groups-nlg/nlg-seminars/
Hosts: Jonathan May and Justin Cho