"Sharing and Abstraction in Hierarchical Reinforcement Learning"

3/24/2000: [time not recorded]
[location not recorded]

Abstract: Reinforcement learning (RL) algorithms learn to solve sequential decision-making tasks by interacting with a Markovian environment. Existing methods treat the Markovian environment as a flat search space. While these methods have successfully solved many interesting problems, there are limits to the size of problem that can be successfully handled as a flat state space. In this talk, we will describe one approach to addressing this problem: Hierarchical reinforcement learning using the MAXQ value function decomposition. In this approach, the programmer defines a hierarchy of tasks and subtasks, and the value function (the key data structure in reinforcement learning) is decomposed hierarchically into value functions for each task and subtask. We will describe how the MAXQ decomposition permits the subtasks to ignore aspects of the state space, and hence, the subtasks become re-usable. This speeds search and learning with only minor losses in optimality.

Last updated: Mon Jun 19 17:44:06 2006