Applied Human-in-the-Loop AI

Friday, October 4, 2019, 11:00 am - 12:00 pm PDTiCal
10th floor conference room (1016)
This event is open to the public.
AI Seminar
Dr. Petar Ristosk, IBM Almaden Research Center

The human-in-the-loop model combines the best of human intelligence with the best of machine intelligence. Machines are good at accurately and efficiently processing large amounts of data, but are not able to make smart decisions if not provided with vast knowledge. On the other hand, humans are inaccurate and slow, but can make decisions with less information. 
Combining them together omits the disadvantages of the individual models and results in a powerful model that can be applied in various AI applications.
Although current research trends claim that fully automatic AI systems will replace human labor in the near future, there are many domains where this is far from reality. One of the first obstacle for this vision is that in many domains near-perfect performance is required. For example, many biomedical applications have near 0% error tolerance, despite datasets full of uncertainty, incompleteness and noise. Furthermore, some problems in the medical domain are quite challenging, making the application of fully automated models difficult, or at least raising questions on the quality of results. Consequently, efficiently including a domain expert as an integral part of the system not only greatly enhances the knowledge discovery process pipeline, but can in certain circumstances be legally or ethically required. 
Furthermore, machines require vast amounts of consistently labeled data to be able to perform well. However, human annotation tasks intrinsically carry a level of disagreement among annotators, regardless of their level of domain expertise. For example, in the medical domain the inter-annotator agreement can be as low as 66%. Thus, the machines cannot learn effectively from such noisy data. While this issue cannot be solved, a human-in-the-loop model can help the subject matter experts to get to the desired solution more efficiently and effectively. 

In this talk I will describe several approaches and algorithms to successfully apply the human-in-the-loop model in real-world applications: (i) Knowledge Extraction; (ii) Cognitive Horizon Surveillance; (iii) Analyzing Data Centers; (iv) Material Discovery; (v) Computational Creativity.

Bio: Dr. Petar Ristoski is a Research Staff Member in the Computer Science Department at the IBM Almaden Research Center. As part of his work, he conducts research in Artificial Intelligence with an emphasis on Neural Networks and Semantic Technologies. His research involves discovering fundamental principles and implementing prototype systems that can be used to understand, analyze, and manage human text as well as collaborating with human experts to train more capable Cognitive Systems.
Dr. Petar Ristoski received his PhD degree in Computer Science from the University of Mannheim, Germany, in 2018. For his PhD thesis he was awarded the SWSA Distinguished Dissertation Award 2019.

« Return to Upcoming Events