Share Email Print

Proceedings Paper

SAN-RL: combining spreading activation networks and reinforcement learning to learn configurable behaviors
Author(s): Daniel M. Gaines; Don Mitchell Wilkes; Kanok Kusumalnukool; Siripun Thongchai; Kazuhiko Kawamura; John H. White
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Reinforcement learning techniques have been successful in allowing an agent to learn a policy for achieving tasks. The overall behavior of the agent can be controlled with an appropriate reward function. However, the policy that is learned will be fixed to this reward function. If the user wishes to change his or her preference about how the task is achieved the agent must be retrained with this new reward function. We address this challenge by combining Spreading Activation Networks and Reinforcement Learning in an approach we call SAN-RL. This approach provides the agent with a causal structure, the spreading activation network, relating goals to the actions that can achieve those goals. This enables the agent to select actions relative to the goal priorities. We combine this with reinforcement learning to enable the agent to learn a policy. Together, these approaches enable the learning of a configurable behaviors, a policy that can be adapted to meet the current preferences. We compare the approach with Q-learning on a robot navigation task. We demonstrate that SAN-RL exhibits goal-directed behavior before learning, exploits the causal structure of the network to focus its search during learning and results in configurable behaviors after learning.

Paper Details

Date Published: 18 February 2002
PDF: 12 pages
Proc. SPIE 4573, Mobile Robots XVI, (18 February 2002); doi: 10.1117/12.457458
Show Author Affiliations
Daniel M. Gaines, Jet Propulsion Lab. (United States)
Don Mitchell Wilkes, Vanderbilt Univ. (United States)
Kanok Kusumalnukool, Vanderbilt Univ. (United States)
Siripun Thongchai, Vanderbilt Univ. (United States)
Kazuhiko Kawamura, Vanderbilt Univ. (United States)
John H. White, Vanderbilt Univ. (United States)

Published in SPIE Proceedings Vol. 4573:
Mobile Robots XVI
Douglas W. Gage; Howie M. Choset, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?