Abstract: Modifying knowledge-based systems (KBSs) is a complex activity. One of its difficulties is that several related portions of the KBS might have to be changed in order to maintain the coherence of the system. However, it is difficult for users to figure out what has to be changed and how. This paper presents a novel approach for building knowledge acquisition (KA) tools that overcomes some of the limitations of current approaches. In this approach, knowledge of prototypical procedures for modifying KBSs is used to guide users in changing all related portions of a KBS. These procedures, which we call knowledge acquisition scripts (or KA Scripts), capture how related portions of a KBS can be changed coordinately. By using KA scripts, a KA tool would be able to relate individual changes in different parts of a KBS, enabling the analysis of each individual change from the perspective of the overall modification. The paper also describes the implementation of ETM: a script-based tool that builds on the EXPECT framework for building KBSs (Gil, 1994), discusses how we have addressed some important issues of this approach, and describes a preliminary empirical evaluation of ETM that shows that KA Scripts allow users to perform KBSs modification tasks more efficiently.
Abstract: Our goal is to build knowledge acquisition tools that support users in modifying knowledge-based systems. These modifications may require several individual changes to various components of the knowledge base, which need to be carefully coordinated to prevent users from leaving the knowledge-based system in an unusable state. This paper describes an approach to building knowledge acquisition tools which capture knowledge about commonly occurring modification sequences and support users in completing the modifications they start. These sequences, which we call KA Scripts, relate individual changes and the effects that they have on the knowledge base. We discuss our experience in designing and compiling a library of KA Scripts. We also describe the implementation of a tool that uses them and our preliminary evaluations that demonstrate their usability.
Abstract: Role-limiting approaches support knowledge acquisition (KA) by centering knowledge base construction on common types of tasks or domain-independent problem-solving strategies. Within a particular problem-solving strategy, domain-dependent knowledge plays specific roles. A KA tool then helps a user to fill these roles. Although role-limiting approaches are useful for guiding KA, they are limited because they only support users in filling knowledge roles that have been built in by the designers of the KA system. EXPECT takes a different approach to KA by representing problem-solving knowledge explicitly, and deriving from the current knowledge base the knowledge gaps that must be resolved by the user during KA. This paper contrasts role-limiting approaches and EXPECT's approach, using the propose-and-revise strategy as an example. EXPECT not only supports users in filling knowledge roles, but also provides support in 1) adapting the problem-solving strategy, 2) changing the types of information to be acquired about a knowledge role, 3) adding new knowledge roles, and 4) acquiring additional background information about the domain needed by the knowledge-based system. EXPECT's guidance changes as the knowledge base changes, providing a more flexible approach to knowledge acquisition. This work provides evidence supporting the need for explicit representations in building knowledge-based systems.
Abstract: EXPECT provides an environment for developing knowledge-based systems that allows end-users to add new knowledge without needing to understand the details of system organization and implementation. The key to EXPECT's approach is that it understands the structure of the knowledge-based system being built: how it solves problems and what knowledge it needs to support problem-solving. EXPECT uses this information to guide users in maintaining the knowledge-based system. We have used EXPECT to develop a tool for evaluating transportation plans.
Abstract: A system that acquires knowledge from a user should be able to reflect upon the knowledge that it has - at each moment - and understand what kinds of new knowledge it needs to learn. For the past two decades, research in the area of knowledge acquisition has been moving towards systems that have access to richer representations of knowledge about their task. This paper reviews some well-known knowledge acquisition tools representative of this trend. It also describes our recent work in EXPECT, a system with explicit representations of knowledge about the task and the domain that supports knowledge acquisition for a wider range of tasks and applications than its predecessors. We hope our observations will be useful to researchers in user interfaces and in machine learning concerned with acquiring information from users.
Abstract: Applications should be an invaluable experimental source of information and challenges for AI research. The real world always stretches the limitations of our AI systems, pointing toward new research themes in many areas. In the process of implementing an application, the designer continually makes choices based on (1) the baseline architecture used to implement the application, (2) the characteristics of the problem itself, or (3) arbitrary decisions and assumptions. All these decisions are intertwined in the resulting application, and as a result, it is not easy to abstract a description of the functionality provided by the architecture itself. At the same time, we would like to base our science on real-world applications that are subject to controllable experiments whose parameters can be modified to obtain experimental results of our programs' behavior. However, real applications rarely facilitate this task. Although the AI community has developed formalisms to describe AI architectures, we are still lacking a formal language to describe tasks and problems that provides a good qualitative understanding of AI applications. We argue that this is a major deficiency that stops feedback from applications to research. This work is an effort towards descriptions of applications in terms that are useful 1) to extract conclusions from particular implementations, 2) to facilitate cross-comparisons among different architectures applied to the same problem, and 3) to facilitate comparisons among different tasks. We analyze the Sisyphus experience, and we propose a set of dimensions to describe applications that distinghish between descriptions of the properties of the architecture, the type of problem, and the data sets. We show how these dimensions can be used to produce useful distinctions in the context of the first Sisyphus task, an office assignment problem. Our hope is that the same dimensions will be useful to other researchers in describing, characterizing, and producing qualitative evaluations of their applications, as well as a useful point of comparison for future Sisyphus efforts.
Abstract: Our goal is to build knowledge acquisition tools that support users in making a broad range of changes to a knowledge base, including both factual and problem-solving knowledge. These changes may require several individual modifications to various parts of the knowledge base, that need to be carefully coordinated to prevent users from introducing errors in the knowledge base. Thus, it becomes essential that our KA tools understand the consequences of each kind of change that the user may initiate, detect any harmful side-effects that can be introduced in the system, and guide the user in resolving them. To address this issue, we have developed a transaction-based approach to knowledge acquisition that can support users in making complex modifications to a knowledge base. A transaction is a sequence of changes that together modify some aspect of the behavior of a knowledge-based system, and that when only partially carried out may leave the knowledge base in an undesirable state. If a user executes a transaction partially, the knowledge acquisition tool must provide guidance to finish it and support the user in achieving the desired modification. This paper also describes our work in extending EXPECT's knowledge acquisition tool to support transaction-based mechanisms. EXPECT tracks the possible problems that arise as a consequence of each individual change to the knowledge base, keeps information about the context of each change, and uses this context to resolve the problems detected and to request the user's intervention if additional information is needed.
Abstract: To create more powerful knowledge acquisition systems, we not only need better acquisition tools, but we need to change the architecture of the knowledge based systems we create so that their structure will provide better support for acquisition. Current acquisition tools permit users to modify factual knowledge but they provide limited support for modifying problem solving knowledge. In this paper, we argue that this limitation (and others) stem from the use of incomplete models of problem solving knowledge and inflexible specification of the interdependencies between problem solving and factual knowledge. We describe the EXPECT architecture which addresses these problems by providing an explicit representation for problem solving knowledge and intent. Using this more explicit representation, EXPECT can automatically derive the interdependencies between problem solving and factual knowledge. By deriving these interdependencies from the structure of the system itself EXPECT supports more flexible and powerful knowledge acquisition.
Abstract: Rapid prototyping and tool reusability have pushed knowledge acquisition research to investigate method-specific knowledge acquisition tools appropriate for predetermined problem-solving methods. We believe that method-dependent knowledge acquisition is not the only approach. The aim of our research is to develop powerful yet versatile machine learning mechanisms that can be incorporated into general-purpose but practical knowledge acquisition tools. This paper shows through examples the practical advantages of this approach. In particular, we illustrate how existing knowledge can be used to facilitate knowledge acquisition through analogy mechanisms within a domain and across domains. Our sample knowledge acquisition dialogues with a domain expert illustrate which parts of the process are addressed by the human and which parts are automated by the tool, in a synergistic cooperation for knowledge-base extension and refinement. The paper also describes briefly the EXPECT problem-solving architecture that facilitates this approach to knowledge acquisition.
Abstract: A knowledge acquisition tool should provide a user with maximum guidance in extending and debugging a knowledge base, by preventing inconsistencies and knowledge gaps that may arise inadvertently. Most current acquisition tools are not very flexible in that they are built for a predetermined inference structure or problem-solving mechanism, and the guidance they provide is specific to that inference structure and hard-coded by their designer. This paper focuses on EXPECT, a reflective architecture that supports knowledge acquisition based on an explicit analysis of the structure of a knowledge-based system, rather than on a fixed set of acquisition guidelines. EXPECT's problem solver is tightly integrated with LOOM, a state-of-the-art knowledge representation system. Domain facts and goals are represented declaratively, and the problem solver keeps records of their functionality within the task domain. When the user corrects the system's knowledge, EXPECT tracks any possible implications of this change in the overall system and cooperates with the user to correct any potential problems that may arise. The key to the flexibility of this knowledge acquisition tool is that it adapts its guidance as the knowledge bases evolve in response to changes introduced by the user.