This paper describes a bio-inspired VISion based actionable INTelligence system (VISINT) that provides automated capabilities to (1) understand objects, patterns, events and behaviors in vision data; (2) translate this understanding into timely recognition of novel and anomalous entities; and (3) discover underlying hierarchies and relationships between disparate labels entered by multiple users to provide a consistent data representation. VISINT is both a system and a novel collection of novel bio-inspired algorithms/modules. These modules can be used independently for various aspects of the actionable intelligence problem or sequenced together for an end-to-end actionable intelligence system. The algorithms can be useful in many other applications such as scene understanding, behavioral analysis, automatic surveillance systems, etc. The bio-inspired algorithms are a novel combination of hierarchical spatial and temporal networks based on the Adaptive Resonance Theory (ART). The novel aspects of this work are that it is an end-to-end system for actionable intelligence that combines existing and novel implementations of various modules in innovative ways to develop a system concept for actionable intelligence. Although there are other algorithms/implementations of several of the modules in VISINT, they suffer from various limitations and often system integration is not considered. The overall VISINT system can be viewed an incremental learning system where no offline training is required and data from multiple sources and times can be seamlessly integrated. The user is in the loop, but due to the semi-supervised nature of the underlying algorithms, only significant variations of entities, not all false alarms, are shown to the user. It does not forget the past even with new learning. While VISINT is designed as a vision-based system, it could also work with other kinds of sensor data that can recognize and locate individual objects in the scene. Beyond that stage of object recognition and localization, all aspects of VISINT are applicable to other kinds of sensor data.

Date Published: 19 May 2009
PDF: 11 pages
Proc. SPIE 7352, Intelligent Sensing, Situation Management, Impact Assessment, and Cyber-Sensing, 73520I (19 May 2009); doi: 10.1117/12.820505

Published in SPIE Proceedings Vol. 7352:
Intelligent Sensing, Situation Management, Impact Assessment, and Cyber-Sensing
Stephen Mott; John F. Buford; Gabriel Jakobson, Editor(s)