Share Email Print

Proceedings Paper

Real-time face and gesture analysis for human-robot interaction
Author(s): Frank Wallhoff; Tobias Rehrl; Christoph Mayer; Bernd Radig
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, or gestures. Facial expressions and gestures are one of the main nonverbal communication mechanisms and pass large amounts of information between human dialog partners. Therefore, to allow for intuitive human-machine interaction, a real-time capable processing and recognition of facial expressions, hand and head gestures are of great importance. We present a system that is tackling these challenges. The input features for the dynamic head gestures and facial expressions are obtained from a sophisticated three-dimensional model, which is fitted to the user in a real-time capable manner. Applying this model different kinds of information are extracted from the image data and afterwards handed over to a real-time capable data-transferring framework, the so-called Real-Time DataBase (RTDB). In addition to the head and facial-related features, also low-level image features regarding the human hand - optical flow, Hu-moments are stored into the RTDB for the evaluation process of hand gestures. In general, the input of a single camera is sufficient for the parallel evaluation of the different gestures and facial expressions. The real-time capable recognition of the dynamic hand and head gestures are performed via different Hidden Markov Models, which have proven to be a quick and real-time capable classification method. On the other hand, for the facial expressions classical decision trees or more sophisticated support vector machines are used for the classification process. These obtained results of the classification processes are again handed over to the RTDB, where other processes (like a Dialog Management Unit) can easily access them without any blocking effects. In addition, an adjustable amount of history can be stored by the RTDB buffer unit.

Paper Details

Date Published: 4 May 2010
PDF: 10 pages
Proc. SPIE 7724, Real-Time Image and Video Processing 2010, 77240A (4 May 2010); doi: 10.1117/12.855963
Show Author Affiliations
Frank Wallhoff, Technische Univ. München (Germany)
Tobias Rehrl, Technische Univ. München (Germany)
Christoph Mayer, Technische Univ. München (Germany)
Bernd Radig, Technische Univ. München (Germany)

Published in SPIE Proceedings Vol. 7724:
Real-Time Image and Video Processing 2010
Nasser Kehtarnavaz; Matthias F. Carlsohn, Editor(s)

© SPIE. Terms of Use
Back to Top