Share Email Print

Proceedings Paper

Grounding natural language commands to StarCraft II game states for narration-guided reinforcement learning
Author(s): Nicholas Waytowich; Sean L. Barton; Vernon Lawhern; Ethan Stump; Garrett Warnell
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longstanding problem of reward sparsity. This is especially true for tasks such as training an agent to play StarCraft II, a real-time strategy game where reward is only given at the end of a game which is usually very long. While this problem can be addressed through reward shaping, such approaches typically require a human expert with specialized knowledge. Inspired by the vision of enabling reward shaping through the more-accessible paradigm of natural-language narration, we investigate to what extent we can contextualize these narrations by grounding them to the goal-specific states. We present a mutual-embedding model using a multi-input deep-neural network that projects a sequence of natural language commands into the same high-dimensional representation space as corresponding goal states. We show that using this model we can learn an embedding space with separable and distinct clusters that accurately maps natural-language commands to corresponding game states . We also discuss how this model can allow for the use of narrations as a robust form of reward shaping to improve RL performance and efficiency.

Paper Details

Date Published: 10 May 2019
PDF: 10 pages
Proc. SPIE 11006, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, 110060S (10 May 2019); doi: 10.1117/12.2519138
Show Author Affiliations
Nicholas Waytowich, U.S. Army Research Lab. (United States)
Sean L. Barton, U.S. Army Research Lab. (United States)
Vernon Lawhern, U.S. Army Research Lab. (United States)
Ethan Stump, U.S. Army Research Lab. (United States)
Garrett Warnell, U.S. Army Research Lab. (United States)

Published in SPIE Proceedings Vol. 11006:
Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications
Tien Pham, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?