Enhanced hyperspectral imaging for urban reconnaissance

A European Defence Agency framework has combined high-spatial and -spectral resolution images from airborne sensors to improve detection and classification of objects in towns and cities. The project offers possible solutions to challenges in obtaining accurate georeferences, and explores the use of change detection algorithms and spectral matching to detect anomalies in the urban environment.

The four-year framework (Detection in Urban scenario using Combined Airborne imaging Sensors, or DUCAS) combined the resources of Sweden, Norway, Germany, The Netherlands, Belgium, France, and Italy. We undertook an extensive field trial in Zeebrugge, Belgium in 2011, supplying instrumentation for 3D mapping, hyperspectral and high-resolution imagery, together with in situ techniques to measure target, background, and atmospheric characterizations. Analyzing data from the trial, we considered many different applications for remote sensing data, reporting preliminary results in 2012.¹

We checked the quality of the remotely sensed hyperspectral data using ground truth measurements (information collected at the site) and modeling of radiative transfer (the movement of radiation with regard to emission, absorption, and scattering). We compared measured spectra extracted from a hyperspectral image with simulated spectra from ground and atmospheric measurements and the MODTRAN computer program (MODerate resolution atmospheric TRANsmission). The measurements included temperature and humidity profiles, target spectral reflectance, surface temperature, and sensor characteristics.

Due to the complexity of urban environments and human activities, the spatial and spectral characteristics of targets and backgrounds are extremely diverse. This makes the detection and the classification of man-made objects very difficult. To overcome this, we used combinations of hyperspectral and high-spatial resolution sensors, sometimes involving an operator. In towns, cities, and industrial areas, spatially unresolved materials are difficult to discriminate based on spectral features alone. We can use high-spatial resolution sensors to classify man-made objects from shape, but there is limited capability for differentiating between materials. Classification based on input from an operator using, for example, high resolution imagery, can improve hyperspectral signal processing in terms of receiver operating characteristics.²

One challenge to mapping the infrastructure of old cities is that much of the roadwork has been performed at different times and with different types of material. To obtain accurate georeferencing, we intersected a high-resolution (10cm) laser-generated digital surface model with the line of sight for every single pixel in the corresponding image.³ Classifying infrastructure involves identifying materials that can be used for more than one purpose, for example concrete and gravel, which are used in roofs as well as roads. A 3D map can mitigate this problem by enabling us to identify structures. Figure 1 shows the road network structure in an old city that incorporates various materials.

Figure 1. Road network in the city of Zeebrugge, Belgium, characterized by three different types of concrete shown in different colors.

Anomaly detection using hyperspectral images⁴ is valuable in applications where concealed targets need to be promptly located with only a single image. However, interpreting or post-processing the detected anomalies requires considerable effort because of the great diversity of materials in the urban environment.¹ Furthermore, using a single image exploits only part of the large amount of information that could be gathered by imaging an area successfully over time, and which combines data from different imaging sensors.

We found that by adopting change detection algorithms applied to multi-temporal images (those taken during more than one time period), allowed us to focus on objects or areas of potential interest. This reduced the need for accurate analysis on every single image, yet increased the capability to discriminate small anomalous changes. We studied change detection using both hyperspectral information and high-resolution imagery in different spectral bands. In Figure 2, we used high resolution imagery in the visible wavelength to obtain data at two different instances and to detect subsequent changes. The performance was highly dependent on small errors at the sub-pixel level.

Figure 2. Change detection using high spatial resolution imagery.

We also studied detection based on spectral matching. We modeled the received radiance given the reflectance signature of a target (the radiation reflected from it), which is based on knowledge of the imaging conditions. This approach, often called forward modeling, is based on a set of simulations with varying atmospheric conditions, illumination contributions, and imaging geometry that produce a set of possible radiance signatures for a given target. We compared these signatures directly to the original hyperspectral image for signature-based target detection, and took into account what is known about the imaging conditions without the need for an exact and fixed atmospheric correction.⁵ We also successfully applied alternative methods, using known materials in the database, such as road materials.

Our findings demonstrate the benefits of using hyperspectral imaging sensors and high-resolution imaging instrumentation for surveillance purposes. The results form a base for follow-on programs by the European Defence Agency, which in future will focus on the performance of multi-sensor systems. These may further improve detection and classification in defense applications.

DUCAS researchers will present methods and results from their work at the SPIE Defense, Security, and Sensing Conference (DSS) in Baltimore 2013.