Spaceborne imaging sensors provide global coverage of the Earth's surface on a routine basis. But ground resolution of multispectral (MS) observations may be inadequate to certain identification tasks, especially those crucial for urban objects. A new generation of satellites, such as Ikonos, QuickBird, and SPOT-5, now offer high-resolution MS and panchromatic (pan) images.

Certain data-fusion techniques, often referred to as data merge^{1} or band sharpening,^{2} take advantage of complementary spatial and spectral resolution characteristics to produce enhanced MS observations. More specifically, pan-sharpened MS images are fusion results in which the MS bands are sharpened via the higher-resolution pan image. In fact, the latter is acquired using the maximum resolution allowed by the imaging sensor, as well as by the datalink throughput. The former, in contrast, are acquired using coarser resolutions, typically, two or four times lower, owing to signal-to-noise-ratio constraints and transmission bottlenecks. After being received at ground stations, the pan image may be merged with the MS data for enhancement.

The goal of pan sharpening is to provide fused bands that closely match the image that would have been be produced by the narrow-band MS sensor if it had the same resolution as the broad-band sensor. Recent methods based on multiresolution analysis (MRA) have shown superior performance. However, data fusion requires defining a model that establishes how the missing high-pass information, which is to be injected into the resampled MS bands, may be extracted from the pan image.

**Pan sharpening with a Kalman filter** Here we describe an algorithm based on Kalman filtering, namely, the wavelet Kalman method (WKM) for pan sharpening. Because injection is conducted on details derived by MRA that are based on the ‘à trous’ wavelet transform (ATWT), the Kalman filter is applied in a multiscale version: that is, the index variable represents scale instead of time. The new algorithm solves some typical problems with recent fusion methods. Unmixing of spatially low resolution pixels becomes possible, and instability in computing the injection gains is eliminated.

The multiscale Kalman filter, which uses a predictive model, allows accurate injection of spatial details from pan that can be objectively assessed in term of mean square error. The vector Kalman filter gains on each MS band regulate the injection of pan details by evaluating the noise power and all prediction errors of MS bands. The greater the value of the error on a specific band, the higher the injection of pan details on that band.

**Description of the algorithm** The ATWT is a nonorthogonal multiresolution decomposition defined by a filter bank. Figure 1 displays a sample image. The levels of ATWT are obtained by separable filtering of the original image. Due to the absence of decimation, the synthesis is simply obtained by summing detail levels to the approximation:

in which *c*_{J}(m,n) and *d*_{j}(m,n), *j*=1,…,*J* are the approximation and details of the analyzed image.

**Figure 1.** ATWT of an original image with *J*=2 levels of decomposition.

The vector-state, scalar-observation Kalman filter addresses the general problem of estimating the *L*×1 vector state, s, of a discrete time process that is governed by the state equation:

and by the observation equation:

Equation (3) relates the state of the measurement *x(n)* by a linear combination of each scalar element of a vector state. The solution to the Kalman filter problem can be solved by the prediction equation

and by the correction and filtering equation:

applying an iterative procedure as illustrated in Figure 2.

**Figure 2.** Iterative procedure for estimating states in the Kalman filter problem.

In the algorithm the vectorial state s[*n*] and the observation *x(n)* are associated with details derived from the ATWT decomposition of MS bands and the pan band. The unknown parameters that have been introduced in Equations (1) and (3) are indirectly determined from the bands by analyzing data at coarser scales in which MS details are known to be in agreement with the ARSIS (a French acronym for ‘enhanced spatial resolution by injection of structures’) concept.^{3} The solution of the multiscale Kalman filter problem is applied at finer scales on MS details to be enhanced as in Equations (4) and (5). Finally, the inverse ATWT is applied to obtain the fused image.

**Experimental results and conclusions** We have tested the proposed algorithms on two very high resolution data sets from Ikonos and QuickBird, acquired on the city of Toulouse, France, and Rome, Italy. The two sets are characterized by four MS bands that span the visible and non-infrared wavelengths with a spatial resolution of 4 and 2.8m, respectively, and the pan image, whose bandwidth covers the wavelengths of the MS images with a spatial resolution of 1 and 0.7m, respectively. Figure 3 show the MS image in a true color composition before and after fusion with the pan image. Visual results confirm that the method is capable of enhancing the spatial quality of MS images while to a great extent preserving their spectral content. Additional fused images can be found on the authors’ Web page,^{4} and quantitative assessment of the fused products can be found in state-of-the-art research.^{5–7}

**Figure 3. **Examples of full-scale spatial enhancement of fusion algorithms displayed as 512×512 true color compositions at full-scale resolution. (a) Original MS image resampled to the scale of a pan image for the Ikonos data set. (b) WKM fusion product for the Ikonos data set. (c) Original MS image resampled to the scale of a pan image for the Quickbird data set. (d) WKM fusion product for the Ikonos data set.

Andrea Garzelli, Filippo Nencini

Department of Information Engineering

University of Siena

Siena, Italy

Andrea Garzelli, PhD, is associate professor of telecommunications at the Department of Information Engineering of the University of Siena, where he currently teaches courses in digital signal processing and remote-sensing systems. His research interests include signal and image analysis, processing, and classification: filtering, SAR image analysis, and image fusion for optical and radar remote-sensing applications. He is a senior member of the IEEE.

Filippo Nencini is a research associate in the Department of Information Engineering at the University of Siena, where he also obtained his PhD in information engineering in 2006. His research interests include wavelet theory and applications to remote sensing: SAR image processing, fractal analysis, multiresolution and multisensor image fusion, and quality assessment of MS image data.