Optimal spectral image fusion for detection of shoreline targets
Spectral-spatial sharpening processes can be used to identify spectral features or anomalies in high-spatial-resolution multispectral (MS) data and in high-spectral-resolution imagery.1–3 Such HS and MS imagery can be used to process pushbroom images that are obtained with the use of inertial motion units and augmented global positioning from airborne or shipborne platforms. These image analysis techniques thus hold much promise for enhancing situational awareness during environmental surveillance and monitoring activities.
In a previously developed technique for the fusion of hyperspectral (HS) and MS signatures, line targets are numerically embedded in the imagery.4 To minimize differences between these high-spatial-resolution HS synthetic signatures and the actual observed HS signatures, however, requires co-registration of the images. This co-registration can be achieved with the use of nearest neighborhood resampling methods, where randomly sampled areas and pixels within the images are selected from the co-registered data. In addition, singular value decomposition5 can be performed on random pairs of pixels in the MS channels and on each HS channel. This mathematical process allows synthetic HS signatures to be ‘predicted.’ Images are thus created for each HS channel from the MS channels, and synthetic HS images with high spatial resolution are produced. To insure that optimal spectral sharpening is achieved, the synthetic HS imagery is processed using low-frequency and high-frequency filtering, i.e., with a 2D discrete cosine and inverse cosine transform6 and a 2D Butterworth filter.7 The use of these (or similar) filters allows spatial features found in high-spatial-resolution MS imagery to be accurately transferred or embedded into the final synthetic HS images (and with higher spatial resolution). In addition, the high-spectral-resolution features can be optimally transferred from the observed low-spatial-resolution HS imagery to the synthetic high-spatial-resolution HS data cubes. A key step in the optimization of this frequency filtering is changing the cutoff and order coefficients of the 2D Butterworth filter. This can be accomplished with the use of interval-halving techniques, in which each band and processed pixel signature is randomly selected so that the original and final sharpened signatures can be compared. The nonparametric Kolmogorov-Smirnov (KS) test5 value can be used to make this comparison. For this test, it is necessary to create ‘cumulative signatures’ of synthetic corrected reflectance,8 radiance, or digital counts versus wavelength (using random samples of pixels and areas). These steps allow the differences between the observed and predicted signatures to be minimized (see Figure 1).
In this work a technique for numerically embedding targets in high-spatial-resolution MS images is demonstrated for the purpose of investigating littoral zone features within images acquired from the shorelines of the Gulf of Mexico.9 In addition, an optimal fusion of HS and MS imagery is performed to obtain new higher spatial and spectral resolution signatures of targets that are obscured by shoreline vegetation. With this approach, it is possible to achieve better detection of difficult targets such as these (and those in the submerged littoral zone) through the use of combined data from different sensor types, image fusion, and simulation techniques.
There are several ways to test the optimization of a spectral-spatial sharpening process. One method, used in this work, is to use line targets or features that are mathematically embedded within the observed high-spatial resolution MS imagery and the associated spectral anomalies in the HS images. In this technique, the HS and MS original channels or bands are spatially and spectrally ‘spiked’ (i.e., embedded) with an artificial feature or target. The MS channels from an area can be modified by embedding a line target (changing the pixel values). In the same way, a spectral anomaly is embedded into the corresponding area of the HS scene. The spectral-spatial sharpening process is then run to determine the optimal filter cutoff, as well as the order coefficients that result from the optimization and the KS signature tests. It is therefore possible to observe and quantify the influence of the optimization procedures by recovering the line target in the new spectrally and spatially sharpened imagery. A single realization of the process—in which the results of embedding imagery for testing optimized spectral-spatial resolution enhancement protocols are demonstrated—is illustrated in Figure 2.
These spectral-spatial sharpening techniques have been applied to images that were collected from aircraft and sea vessels.10–12 The HS and MS images that were collected simultaneously are shown in Figure 2. These images were obtained from a vessel that was close to a shoreline with known anomalies (i.e., weathered oil features) at the water–land interface. After the sharpening process on these images was complete, the suspected anomalies could be analyzed with the use of optimized higher-order derivative spectroscopy13 and optimal multi-wavelength contrast algorithms. Optimal translating and dilating wavelet analysis11 of the synthetic HS signatures was also conducted.
Other examples in which this approach is demonstrated are shown in Figures 3 and 4. For both these cases the spectral-spatial sharpening operations, for feature detection of weathered oil on shorelines, were used. In Figure 3, the HS imagery (middle) has a ground sampling distance (GSD) of about 5cm. With the use of the MS digital image shown on the left (with a GSD of about 5mm), the middle image was sharpened. This spectral-spatial sharpening of the HS image cube's spectra enhanced the feature detections (i.e., of the weathered oil residue). As such, this methodology has been successfully demonstrated as an effective alternative to visual inspections and surveillance. Airborne and vessel HS pushbroom sensor data can also be corrected,14 with the use of an inertial motion unit.15–17 An optimized airborne three-channel MS-HS sharpened image is shown in Figure 4. The sharpened image illustrates the presence of submerged weathered oil features along Florida's Gulf of Mexico ‘panhandle’ shoreline. This data was acquired with a GSD of about 10cm, from an altitude of about 1500m.
In summary, a technique for achieving an optimal fusion of hyperspectral and multispectral imagery, in which line targets are numerically embedded into acquired images, has been developed. The differences between high-spatial-resolution synthetic and observed spectral signatures are also minimized. Images that were obtained from airborne and shipborne platforms have been used to demonstrate the power of this technique for target detection along shorelines. An area of ongoing work involves the use of the optimized HS-MS image fusion approach for studying in situ colloidal aggregates and flocs within lutoclines and moving fluid mud layers.
College of Engineering
Charles Bostater is a professor and director of the Center for Remote Sensing and Marine Environmental Optics Laboratory. He teaches courses on the physics of the ocean and atmosphere, remote sensing, and geographic information systems. He has built airborne hyperspectral imaging systems that he operates from in situ platforms, goniometers, and vessels.