Location: CR#1135-#1137 Conference Rooms ISI-MDR

Speaker: Tejas Srinavasan, USC

Abstract: With the proliferation of AI assistants in many high stakes and safety-critical decision-making tasks, it is important to understand what factors modulate how people rely on AI assistance. One important factor is the user’s trust in the AI assistant, with low and high levels of trust resulting in users ignoring accurate AI advice (under-reliance) and accepting incorrect AI advise (over-reliance), respectively. We propose that AI assistants should adapt their behavior through trust-adaptive interventions to mitigate such inappropriate reliance. For instance, when user trust is low, providing an explanation can elicit more careful consideration of the assistant’s advice by the user. In two decision-making scenarios — laypeople answering science questions and doctors making medical diagnoses — we find that providing supporting and counter-explanations during moments of low and high trust, respectively, yields up to 38% reduction in inappropriate reliance and 20% improvement in decision accuracy. We are similarly able to reduce over-reliance by adaptively inserting forced pauses to promote deliberation when users have high trust in the AI assistant. Our results highlight how AI adaptation to user trust can facilitate appropriate reliance, presenting exciting avenues for improving human-AI collaboration.