Event Details

Speaker: Xuhui Zhou, Carnegie Mellon University
Location: virtually online.

January 17, 2025

Hosted by: Abel Salinas
POC: Maura Covaci

LLMs are increasingly playing agentic and social roles by interacting with users and tools. I’ll first introduce SOTOPIA, a framework for evaluating LLM social intelligence through interactive simulations. Showing that LLMs struggle with real-world social skills, I will then discuss the crucial role of information asymmetry, highlighting that LLMs cannot properly deal with the information asymmetry that is present in real interactions. Finally, I will introduce HAICosystem, a new framework for simulating and evaluating the safety of LLM agents in user-AI-tool interactions, showcasing the many safety issues that could arise from increasingly autonomous and tool-using agents. I will conclude with some future directions towards safe and socially intelligent LLM agents.

This event will be recorded.

It will be posted on our USC/ISI YouTube page within 1-2 business days: http://www.youtube.com/user/USCISI.

Speaker Bio

Xuhui Zhou is a third-year PhD student in Carnegie Mellon University’s Language Technologies Department (CMU LTI) advised by Maarten Sap. His research focuses on making NLP systems socially intelligent, aligning them with human social values. He has presented his work at top-tier NLP and AI conferences, receiving spotlights, and runner-up best paper at ICLR 2024, and EmeCom 2022. Additionally, he has been red-teaming for GPT-4. His work has been covered in the press, including The New York Times, Financial Times, and Marktechpost. He previously organized the workshop on Theory of Mind in Communicating Agents at ICML 2023 and he has served as a reviewer for major conferences, including NeurIPS, ICLR, ACL, EMNLP, etc.