Share Email Print
cover

Proceedings Paper

Multimodally controlled intelligent telerobot for people with disabilities
Author(s): Zunaid Kazi; Shoupu Chen; Matthew Beitler; Daniel Chester; Richard Foulds
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

This paper reports on the current status of the multimodal user supervised interface and intelligent control (MUSIIC) project, which is working towards the development of an intelligent assistive telemanipulative system for people with motor disabilities. Our MUSIIC strategy overcomes the limitations of previous approaches by integrating a multimodal RUI (robot user interface) and a semi-autonomous reactive planner that will allow users with severe motor disabilities to manipulate objects in an unstructured domain. The multimodal user interface is a speech and deictic (pointing) gesture based control that guides the operation of a semi-autonomous planner controlling the assistive telerobot. MUSIIC uses a vision system to determine the three-dimensional shape, pose and color of objects and surfaces which are in the environment, and as well as an object-oriented knowledge base and planning system which superimposes information about common objects in the three-dimensional world. This approach allows the users to identify objects and tasks via a multimodal user interface which interprets their deictic gestures and a restricted natural language like speech input. The multimodal interface eliminates the need for general purpose object recognition by binding the users speech and gesture input to a locus in the domain of interest. The underlying knowledge-driven planner, combines information obtained from the user, the stereo vision mechanism as well as the knowledge bases to adapt previously learned plans to perform new tasks and also to manipulate newly introduced objects into the workspace. Therefore, what we have is a flexible and intelligent telemanipulative system that functions as an assistive robot for people with motor disabilities.

Paper Details

Date Published: 17 December 1996
PDF: 12 pages
Proc. SPIE 2901, Telemanipulator and Telepresence Technologies III, (17 December 1996); doi: 10.1117/12.263001
Show Author Affiliations
Zunaid Kazi, Univ. of Delaware and A. I. duPont Hospital for Children (United States)
Shoupu Chen, Univ. of Delaware and A. I. duPont Hospital for Children (United States)
Matthew Beitler, Univ. of Delaware and A. I. duPont Hospital for Children (United States)
Daniel Chester, Univ. of Delaware and A. I. duPont Hospital for Children (United States)
Richard Foulds, Univ. of Delaware and A. I. duPont Hospital for Children (United States)


Published in SPIE Proceedings Vol. 2901:
Telemanipulator and Telepresence Technologies III
Matthew R. Stein, Editor(s)

© SPIE. Terms of Use
Back to Top