Share Email Print

Proceedings Paper

Multimodal interaction for human-robot teams
Author(s): Dustin Burke; Nathan Schurr; Jeanine Ayers; Jeff Rousseau; John Fertitta; Alan Carlin; Danielle Dumond
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Unmanned ground vehicles have the potential for supporting small dismounted teams in mapping facilities, maintaining security in cleared buildings, and extending the team’s reconnaissance and persistent surveillance capability. In order for such autonomous systems to integrate with the team, we must move beyond current interaction methods using heads-down teleoperation which require intensive human attention and affect the human operator’s ability to maintain local situational awareness and ensure their own safety. This paper focuses on the design, development and demonstration of a multimodal interaction system that incorporates naturalistic human gestures, voice commands, and a tablet interface. By providing multiple, partially redundant interaction modes, our system degrades gracefully in complex environments and enables the human operator to robustly select the most suitable interaction method given the situational demands. For instance, the human can silently use arm and hand gestures for commanding a team of robots when it is important to maintain stealth. The tablet interface provides an overhead situational map allowing waypoint-based navigation for multiple ground robots in beyond-line-of-sight conditions. Using lightweight, wearable motion sensing hardware either worn comfortably beneath the operator’s clothing or integrated within their uniform, our non-vision-based approach enables an accurate, continuous gesture recognition capability without line-of-sight constraints. To reduce the training necessary to operate the system, we designed the interactions around familiar arm and hand gestures.

Paper Details

Date Published: 17 May 2013
PDF: 17 pages
Proc. SPIE 8741, Unmanned Systems Technology XV, 87410E (17 May 2013); doi: 10.1117/12.2016334
Show Author Affiliations
Dustin Burke, Aptima, Inc. (United States)
Nathan Schurr, Aptima, Inc. (United States)
Jeanine Ayers, Aptima, Inc. (United States)
Jeff Rousseau, Aptima, Inc. (United States)
John Fertitta, Aptima, Inc. (United States)
Alan Carlin, Aptima, Inc. (United States)
Danielle Dumond, Aptima, Inc. (United States)

Published in SPIE Proceedings Vol. 8741:
Unmanned Systems Technology XV
Robert E. Karlsen; Douglas W. Gage; Charles M. Shoemaker; Grant R. Gerhart, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?