Share Email Print

Proceedings Paper

Biologically-inspired robust and adaptive multi-sensor fusion and active control
Author(s): Deepak Khosla; Paul A. Dow; David J. Huber
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

In this paper, we describe a method and system for robust and efficient goal-oriented active control of a machine (e.g., robot) based on processing, hierarchical spatial understanding, representation and memory of multimodal sensory inputs. This work assumes that a high-level plan or goal is known a priori or is provided by an operator interface, which translates into an overall perceptual processing strategy for the machine. Its analogy to the human brain is the download of plans and decisions from the pre-frontal cortex into various perceptual working memories as a perceptual plan that then guides the sensory data collection and processing. For example, a goal might be to look for specific colored objects in a scene while also looking for specific sound sources. This paper combines three key ideas and methods into a single closed-loop active control system. (1) Use high-level plan or goal to determine and prioritize spatial locations or waypoints (targets) in multimodal sensory space; (2) collect/store information about these spatial locations at the appropriate hierarchy and representation in a spatial working memory. This includes invariant learning of these spatial representations and how to convert between them; and (3) execute actions based on ordered retrieval of these spatial locations from hierarchical spatial working memory and using the "right" level of representation that can efficiently translate into motor actions. In its most specific form, the active control is described for a vision system (such as a pantilt- zoom camera system mounted on a robotic head and neck unit) which finds and then fixates on high saliency visual objects. We also describe the approach where the goal is to turn towards and sequentially foveate on salient multimodal cues that include both visual and auditory inputs.

Paper Details

Date Published: 13 April 2009
PDF: 9 pages
Proc. SPIE 7345, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2009, 73450J (13 April 2009); doi: 10.1117/12.820503
Show Author Affiliations
Deepak Khosla, HRL Labs., LLC (United States)
Paul A. Dow, HRL Labs., LLC (United States)
David J. Huber, HRL Labs., LLC (United States)

Published in SPIE Proceedings Vol. 7345:
Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2009
Belur V. Dasarathy, Editor(s)

© SPIE. Terms of Use
Back to Top