This PDF file contains the front matter associated with SPIE Proceedings Volume 7710, including the Title Page, Copyright information, Table of Contents, and the Conference Committee listing.

Many problems in electromagnetic signal analysis exhibit dynamics on a wide range of time scales. Further, these dynamics may involve both continuous source generation processes and discrete source mode dynamics. These rich temporal characteristics can present challenges for standard modeling approaches, particularly in the presence of nonstationary noise and clutter sources. Here we demonstrate a hybrid algorithm designed to capture the dynamic behavior at all relevant time scales while remaining robust to clutter and noise at each time scale. We draw from techniques of adaptive feature extraction, statistical machine learning, and discrete process modeling to construct our hybrid algorithm. We describe our approach and present results applying our hybrid algorithm to a simulated dataset based on an example radio beacon identification problem: civilian air traffic control. This application illustrates the multi-scale complexity of the problems we wish to address. We consider a multi-mode air traffic control radar emitter operating against a cluttered background of competing radars and continuous-wave communications signals (radios, TV broadcasts). Our goals are to find a compact representation of the radio frequency measurements, identify which pulses were emitted by the target source, and determine the mode of the source.

This paper discusses some of the techniques developed at MIT Lincoln Laboratory for information fusion of lidar-based biological standoff sensors, meteorology, point sensors, and potentially other information sources, for biodefense applications. The developed Spatiotemporal Coherence (STC) fusion approach includes phenomenology aspects and approximate uncertainty measures for information corroboration quantification. A supervised machine-learning approach was also developed. Computational experiments involved ground-truth data generated from measurements and by simulation techniques that were developed. The fusion results include performance measures that focus explicitly on the fusion algorithms' effectiveness. Both fusion approaches enable significant false-alarm reduction. Their respective advantages and tradeoffs are examined.

High resolution LiDAR data is used to augment spectral data to improve resolution/accuracy. Digital elevation information, texture information, and spectral data are all combined into a single dataset and different clustering algorithms are used on the raster information and compared with clusters of spectral data alone. Long term goals of the work are to find efficient and effective methods of combining different data sets of varying resolution from different sources into a single dataset for analysis to improve data and classification resolution and accuracy.

Hybrid Intrinsic Cellular Inference Network (HICIN) is designed for battlespace decision support applications. We developed an automatic method of generating hypotheses for an entity-attribute classifier. The capability and effectiveness of a domain specific ontology was used to generate automatic categories for data classification. Heterogeneous data is clustered using an Adaptive Resonance Theory (ART) inference engine on a sample (unclassified) data set. The data set is the Lahman baseball database. The actual data is immaterial to the architecture, however, parallels in the data can be easily drawn (i.e., "Team" maps to organization, "Runs scored/allowed" to Measure of organization performance (positive/negative), "Payroll" to organization resources, etc.). Results show that HICIN classifiers create known inferences from the heterogonous data. These inferences are not explicitly stated in the ontological description of the domain and are strictly data driven. HICIN uses data uncertainty handling to reduce errors in the classification. The uncertainty handling is based on subjective logic. The belief mass allows evidence from multiple sources to be mathematically combined to increase or discount an assertion. In military operations the ability to reduce uncertainty will be vital in the data fusion operation.

We address the problem of fusing ESM reports by two evidential reasoning schemes, namely Dempster-Shafer theory and Dezert-Smarandache theory. These schemes provide results in different frames of discernment, but are able to fuse realistic ESM data. We discuss their advantages and disadvantages under varying conditions of sensor data certainty and fusion reliability, the latter coming from errors in the association process. A thresholded version of Dempster-Shafer theory is fine-tuned for performance across a wide range of values for certainty and reliability. The results are presented first for typical scenarios, and secondly for Monte-Carlo studies of scenarios under varying sensor certainty and fusion reliability. The results exhibit complex non-linear functions, but for which clear trends can nevertheless be extracted. A compromise has to be achieved between stability under occasional miss-associations, and reaction time latency under a real change of allegiance. The alternative way of reporting results through Dezert-Smarandache theory is studied under similar conditions, and shown to provide good results, which are however more dependent on the unreliability, and slightly less stable. In this case however, the frame of discernment is larger, and permits additional interpretations, which are outside the scope of Dempster-Shafer.

The JDL model for fusion provides a structure for fusion of multispectral data at all levels. Fused data provides improved performance in Automatic Target Recognition (ATR). Critical to the overall fusion performance, however, is the low level(0-2) fusion of sensory and context information. Loss of information must be avoided at this level, but complexity must be reduced. A model is presented that uses fuzzy sets to form entities and capture the information needed for target recognition. Examples using multi-spectral imagery will be presented.

This paper presents a Multi-Layered Context Impact Modulation (MCIM) technique for persistent surveillance systems (PSS) and discusses its layered architecture for different context modulations including: spatial, temporal, sensor reliability, human presence, and environmental modulations. This paper also presents a fusion model for enhancement of focus of attention at the common operation picture (COP). The fusion model combines all the impacts from the different MCIM layers onto one unified modulated map. To test and evaluate the performance of MCIM, several experiments were conducted to modulate interaction of humans and vehicles which exhibit various normal and suspicious behaviors. The experimental results show strength of this approach in correctly modulating different suspicious situations with higher degree of certainty.

Information fusion is essential for the retrieval of desired information in a sufficiently precise, complete, and robust manner. The Bayesian approach provides a powerful and mathematically funded framework for information fusion. By local Bayesian fusion approaches, the computational complexity of Bayesian fusion gets drastically reduced. This is done by a concentration of the actual fusion task on its probably most task relevant aspects. In this contribution, further research results on a special local Bayesian fusion technique called focussed Bayesian fusion are reported. At focussed Bayesian fusion, the actual Bayesian fusion task gets completely restricted to the probably most relevant parts of the range of values of the Properties of Interest. The practical usefulness of focussed Bayesian fusion is shown by the use of an example from the field of reconnaissance. Within this example, final decisions are based on local significance considerations and consistency arguments. As shown in previous publications, the absolute values of focussed probability statements represent upper bounds for their global values. Now, lower bounds which are obtained from the knowledge about the construction of the focussed Bayesian model are proven additionally. The usefulness of the resulting probability interval scheme is discussed.

A sensor network based on the LIDAR (LIght Detection And Ranging) principle is investigated in order to track persons inside a surveillance area and be able to identify security-relevant behavior. In order to minimize the overall sensor network complexity, power consumption and costs, we recently investigated the network topology based on a quality measure in terms of the number of nodes, measurement distance, width of the LIDAR beams and localization as well as classification performance. As a result, stationary beams with rather small opening angles of up to a few degrees are a good compromise. Since certain regions of the surveillance area are not directly assessed by the LIDAR beams, the accurate tracking of a target throughout the entire area of surveillance is challenging. We demonstrated that tracking based on a Kalman filter approach can nevertheless deliver satisfactory results for a single person inside the surveillance area. In this paper we focus on the task of reliably tracking two persons inside the surveillance area at the same time. The tracking of multiple moving targets is carried out by applying a Multiple Hypothesis Tracking filter approach. The localization and tracking performances are derived from simulations and experiments carried out with commercial laser scanners. In the near future, the rather expensive laser scanners will be replaced by appropriate LIDAR range finders.

In this paper, we consider the problem of detecting the presence of footsteps using signal measurements from a network of seismic sensors. Since the sensors are closely spaced, they result in correlated measurements. A novel method for detection that exploits the spatial dependence of sensor measurements using copula functions is proposed. An approach for selecting the copula function that is most suited for modeling the spatial dependence of sensor observations is also provided. The performance of the proposed approach is illustrated using real footstep signals collected using an experimental test-bed consisting of seismic sensors.

Cognitive robots in the context of space exploration are envisioned with advanced capabilities of model building, continuous planning/re-planning, self-diagnosis, as well as the ability to exhibit a level of 'understanding' of new situations. An overview of some JPL components (e.g. CASPER, CAMPOUT) and a description of the architecture CARACaS (Control Architecture for Robotic Agent Command and Sensing) that combines these in the context of a cognitive robotic system operating in a various scenarios are presented. Finally, two examples of typical scenarios of a multi-robot construction mission and a human-robot mission, involving direct collaboration with humans is given.

As robots become more common, it becomes increasingly useful for many applications to use them in teams that sense the world in a distributed manner. In such situations, the robots or a central control center must communicate and fuse information received from multiple sources. A key challenge for this problem is perceptual heterogeneity, where the sensors, perceptual representations, and training instances used by the robots differ dramatically. In this paper, we use Gärdenfors' conceptual spaces, a geometric representation with strong roots in cognitive science and psychology, in order to represent the appearance of objects and show how the problem of heterogeneity can be intuitively explored by looking at the situation where multiple robots differ in their conceptual spaces at different levels. To bridge low-level sensory differences, we abstract raw sensory data into properties (such as color or texture categories), represented as Gaussian Mixture Models, and demonstrate that this facilitates both individual learning and the fusion of concepts between robots. Concepts (e.g. objects) are represented as a fuzzy mixture of these properties. We then treat the problem where the conceptual spaces of two robots differ and they only share a subset of these properties. In this case, we use joint interaction and statistical metrics to determine which properties are shared. Finally, we show how conceptual spaces can handle the combination of such missing properties when fusing concepts received from different robots. We demonstrate the fusion of information in real-robot experiments with a Mobile Robots Amigobot and Pioneer 2DX with significantly different cameras and (on one robot) a SICK lidar.ÿÿÿÿ

One of the objectives of Cognitive Robotics is to construct robot systems that can be directed to achieve realworld goals by high-level directions rather than complex, low-level robot programming. Such a system must have the ability to represent, problem-solve and learn about its environment as well as communicate with other agents. In previous work, we have proposed ADAPT, a Cognitive Architecture that views perception as top-down and goaloriented and part of the problem solving process. Our approach is linked to a SOAR-based problem-solving and learning framework. In this paper, we present an architecture for the perceptive and world modelling components of ADAPT and report on experimental results using this architecture to predict complex object behaviour. A novel aspect of our approach is a 'mirror system' that ensures that the modelled background and foreground objects are synchronized with observations and task-based expectations. This is based on our prior work on comparing real and synthetic images. We show results for a moving object that collides and rebounds from its environment, hence showing that this perception-based problem solving approach has the potential to be used to predict complex object motions.

This paper describes our work on integrating distributed, concurrent control in a cognitive architecture, and using it to classify perceived behaviors. We are implementing the Robot Schemas (RS) language in Soar. RS is a CSP-type programming language for robotics that controls a hierarchy of concurrently executing schemas. The behavior of every RS schema is defined using port automata. This provides precision to the semantics and also a constructive means of reasoning about the behavior and meaning of schemas. Our implementation uses Soar operators to build, instantiate and connect port automata as needed. Our approach is to use comprehension through generation (similar to NLSoar) to search for ways to construct port automata that model perceived behaviors. The generality of RS permits us to model dynamic, concurrent behaviors. A virtual world (Ogre) is used to test the accuracy of these automata. Soar's chunking mechanism is used to generalize and save these automata. In this way, the robot learns to recognize new behaviors.

This paper will detail the progress on the development of the Symbolic and Subsymbolic Robotics Intelligence Control System (SS-RICS). The system is a goal oriented production system, based loosely on the cognitive architecture, the Adaptive Control of Thought-Rational (ACT-R) some additions and changes. We have found that in order to simulate complex cognition on a robot, many aspects of cognition (long term memory (LTM), perception) needed to be in place before any generalized intelligent behavior can be produced. In working with ACT-R, we found that it was a good instantiation of working memory, but that we needed to add other aspects of cognition including LTM and perception to have a complete cognitive system. Our progress to date will be noted and the challenges that remain will be addressed.

The Cognitive Robotic System (CRS) has been developed to use the Soar cognitive architecture for the control of unmanned vehicles and has been tested on two heterogeneous ground robots: a six-legged robot (hexapod) and a wheeled robot. The CRS has been used to demonstrate the applicability of Soar for unmanned vehicles by using a Soar agent to control a robot to navigate to a target location in the presence of a cul-de-sac obstacle. Current work on the CRS has focused on the development of computer vision, additional sensors, and map generating systems that are capable of generating high level information from the environment that will be useful for reasoning in Soar. The scalability of Soar allows us to add more sensors and behaviors quite easily.

Multiscale fusion algorithms (wavelet or pyramid) can generally satisfy multisensory image fusion. However, those algorithms are not ideal to fuse visible images and infrared images whose intensities appear inverted. Therefore, a novel orientation-based fusion algorithm is proposed in this paper to address this problem. Specifically, a set of MxN Gabor wavelet transforms (GWT) are performed with two input images (I_A and I_B). At each frequency band (b = 1, 2, ..., M), the index of maximal GWT magnitude between two images is selected pixel by pixel; and then two index frequencies, H_A(b) and H_B(b), are calculated as its index accumulation along N orientations, respectively. The final H_A and H_B are the weighted summations through M bands, where the band weights (W_b) are given empirically. Eventually, the fused image is computed as I_F = (I_A .* H_A + IB .* H_B)/( H_A + H_B), where '.*' denotes element-by-element product of two arrays. The orientation-based fusion algorithm can be further varied by either keeping DC (direct current) or suppressing DC in GWT. "Keeping DC" will produce a contrast-smooth image; while "suppressing DC" will result a sharpened fusion. Color fusion is achieved by replacing the red channel of a color image with the fused image, which is suitable for poorly illuminated color images. Not only are the fused images of visible and infrared images satisfied, but the fusions of other image sets are also comparable to the results of multiscale fusion algorithms. The proposed algorithm can be applied to multiple (more than two) image fusion.

Multiple-hypothesis tracking (MHT) algorithms solve the report-to-track association problem^1-5 by accumulating kinematic evidence provided by one or more sensors over time to find likely correlations in the data. MHT technologies have long been applied to such problems as machine vision; automatic target tracking using radar moving target indicator (MTI) for ground, air, sea, and space vehicles; and video-based object tracking. However, relying on kinematic information alone to maintain reliable track in a multi-target scenario is problematic due to a plethora of issues such as sensor limitations, obscuration of the targets, or target/confuser proximity.^6-7 We present our track fusion algorithm, the Long-term Hypothesis Tree (LTHT) that solves the tracklet-to-tracklet association problem by using signature information to repair errors in kinematic tracks. LTHT provides a framework for using arbitrary target signatures such as spectral or shape characteristics to correct errors made by an MHT tracker. The LTHT represents high-level interactions among complex tracks by condensing kinematic track trajectories into a compact representation that can be efficiently maintained over much longer temporal scales than typical MHT trees.

Recent developments in image fusion give the user community many options for ways of presenting the imagery to an end-user. Individuals at the US Army RDECOM CERDEC Night Vision and Electronic Sensors Directorate have developed an electronic system that allows users to quickly and efficiently determine optimal image fusion algorithms and color parameters based upon collected imagery and videos from environments that are typical to observers in a military environment. After performing multiple multi-band data collections in a variety of military-like scenarios, different waveband, fusion algorithm, image post-processing, and color choices are presented to observers as an output of the fusion system. The observer preferences can give guidelines as to how specific scenarios should affect the presentation of fused imagery.

The Sarnoff Acadia® II is a powerful vision processing SoC (System-on-a-Chip) that was specifically developed to support advanced vision applications where system size, weight and/or power are severely constrained. This paper, targeted at vision system developers, presents a detailed technical overview of the Acadia® II, highlighting its architecture, processing capabilities, memory and peripheral interfaces. All major subsystems will be covered, including: video preprocessing, specialized vision processing cores for multi-spectral image fusion, multi-resolution contrast normalization, noise coring, image warping, and motion estimation. Application processing via the MPCore®, an integrated set of four ARM®11 floating point processors with associated peripheral interfaces is presented in detail. The paper will emphasize the programmability of the Acadia® II, while describing its ability to provide state-of-the-art realtime image processing in a small, power optimized package.*

In many instances, sensing tasks are best addressed with multiple sensing modalities. However, fusion of the outputs of disparate sensor systems presents a significant challenge to forming a cohesive sensing system. A discussion of strategies for fusion of disparate sensor data is presented and illustrated with examples of real time and retrospective data fusion for multisensor systems. The first example discussed is a real-time system for situational awareness and the detection of damage control events in ship compartments. The second example is a retrospective data fusion framework for a multisensor system for the detection of buried unexploded ordnance at former bomb and target ranges.

A number of cyber security technologies have proposed the use of data fusion to enhance the defensive capabilities of the network and aid in the development of situational awareness for the security analyst. While there have been advances in fusion technologies and the application of fusion in intrusion detection systems (IDSs), in particular, additional progress can be made by gaining a better understanding of a variety of data fusion processes and applying them to the cyber security application domain. This research explores the underlying processes identified in the Joint Directors of Laboratories (JDL) data fusion process model and further describes them in a cyber security context.

We present a computationally efficient color image fusion algorithm for merging infrared and visible images. At the core of the proposed method is the color transfer technique based on the linear YC_BC_R space. The method directly uses the grayscale fused image and the difference signals of the input images to construct the source YCBCR components, then uses the statistical color transfer technique to form a color fused image that takes the target image's color characteristics. Two different strategies, which respectively employ the pixel averaging fusion scheme and the multiresolution fusion scheme as the grayscale image fusion solution, are proposed to fulfill different user needs. The simple strategy using the pixel averaging fusion scheme answers to a need of easy implementation and speed of use. And the complex strategy using the multiresolution fusion scheme answers to the high quality need of the fused products. In addition, we also describe some useful theories about color-transfer-based image fusion. Experimental results show that the proposed color image fusion algorithm can effectively produce a natural appearing "daytime-like" color fused image, and even using the pixel averaging fusion scheme to implement the grayscale fusion can also provide a pleasing result.

Vision system designers often face the daunting challenge of implementing powerful image processing capabilities in severely size, weight and power constrained systems. Multi-sensor fusion, image stabilization, image enhancement, target detection and object tracking are fundamental processing techniques required by UAVs (Unmanned Aerial Vehicles), smart cameras, weapon sights, and vehicle situational awareness systems. All of these systems also process non-vision data while communicating large amounts of information elsewhere. To meet their demanding requirements, Sarnoff developed the Acadia® II System-on-a-Chip, combining dedicated image processing cores, four ARM®11 processors and an abundance of peripherals in a single Integrated Circuit. This paper will describe how to best use the power of the Acadia® II as both an all-in-one image processor and as a general purpose computer for performing other critical non-vision tasks, such as flight control and system-to-system communication.