Abstract

AI systems planned for deployment in real-world applications are frequently researched and developed in closed simulation environments where all variables are controlled and known to the simulator or using labeled benchmark datasets. Transition from these simulators, testbeds, and benchmark datasets to more openworld domains poses significant challenges to AI systems, including significant increases in the complexity of the domain and the inclusion of real-world novelties where the open-world environment contains numerous out-of-distribution elements that are not part in the AI systems' training set. We propose here a path to a general, domain-independent measure of domain complexity level. We distinguish two aspects of domaincomplexity: intrinsic and extrinsic. The intrinsic complexity is the complexity that exists by itself without any action or interaction from an AI agent, who is performing a task on that domain. This is an agentindependent aspect of the domain complexity. The extrinsic domain complexity is agent-and task-dependent. Intrinsic and extrinsic elements combined capture the overall complexity of the domain. We frame the components that define and impact domain-complexity levels in a domain-independent light. Domain-independent measures of complexity could enable quantitative predictions of the difficulty posed to AI systems when transitioning from one testbed or environment to another, when facing out-of-distribution data in openworld tasks, and when navigating the rapidly expanding solution and search spaces encountered in open-worlds.

Date

2023

Authors

Mayank Kejriwal

Journal

arXiv (Cornell University)