Share Email Print

Proceedings Paper

Camera assisted multimodal user interaction
Author(s): Jari Hannuksela; Olli Silvén; Sami Ronkainen; Sakari Alenius; Markku Vehviläinen
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Since more processing power, new sensing and display technologies are already available in mobile devices, there has been increased interest in building systems to communicate via different modalities such as speech, gesture, expression, and touch. In context identification based user interfaces, these independent modalities are combined to create new ways how the users interact with hand-helds. While these are unlikely to completely replace traditional interfaces, they will considerably enrich and improve the user experience and task performance. We demonstrate a set of novel user interface concepts that rely on built-in multiple sensors of modern mobile devices for recognizing the context and sequences of actions. In particular, we use the camera to detect whether the user is watching the device, for instance, to make the decision to turn on the display backlight. In our approach the motion sensors are first employed for detecting the handling of the device. Then, based on ambient illumination information provided by a light sensor, the cameras are turned on. The frontal camera is used for face detection, while the back camera provides for supplemental contextual information. The subsequent applications triggered by the context can be, for example, image capturing, or bar code reading.

Paper Details

Date Published: 27 January 2010
PDF: 9 pages
Proc. SPIE 7542, Multimedia on Mobile Devices 2010, 754203 (27 January 2010); doi: 10.1117/12.839449
Show Author Affiliations
Jari Hannuksela, Univ. of Oulu (Finland)
Olli Silvén, Univ. of Oulu (Finland)
Sami Ronkainen, Nokia Devices (Finland)
Sakari Alenius, Nokia Research Ctr. (Finland)
Markku Vehviläinen, Nokia Research Ctr. (Finland)

Published in SPIE Proceedings Vol. 7542:
Multimedia on Mobile Devices 2010
Reiner Creutzburg; David Akopian, Editor(s)

© SPIE. Terms of Use
Back to Top