Abstract

Building machines capable of common sense reasoning is an important milestone in achieving Artificial General Intelligence (AGI). While recent advances, such as large language models, are promising, systematic and sufficiently robust evaluations of these models on common sense have been inadequate, and designed for an earlier generation of models. One criticism of prior evaluation protocols is that they have been too narrow in scope e.g., by restricting the format of questions posed to the model, not being theoretically grounded, and not taking the context of a model’s responses in constructing follow-up questions or asking for explanations. In this paper, we aim to address this gap by proposing a context-rich evaluation protocol designed specifically for evaluating machine common sense. Our protocol can subsume popular evaluation paradigms in machine common sense as special cases, and is suited for …

Date

May 24, 2023

Authors

Mayank Kejriwal, Henrique Santos, Ke Shen, Alice M Mulvehill, Deborah L McGuinness

Book

International Conference on Artificial General Intelligence

Pages

167-176

Publisher

Springer Nature Switzerland