A hybrid Pan-sharpening approach using maximum local extrema

ABSTRACT


INTRODUCTION
Most of remote sensing satellites are producing images with high spectral and low spatial or low spectral and high spatial resolutions i.e. Multispectral and Panchromatic images respectively. Sharp spectral resolution highlights different objects present in the image while main feature of accurate spatial resolution is to describe the outline, contents and structure of the image. Neither a multispectral nor a panchromatic image is sufficient to best describe the earth images essential for many of the remote sensing applications. High spatial and spectral images are suitable for observing the earth images as it involves information like feature, shape, structure and different objects. Direct production of such images is inconvenient due to lack of bandwidth and onboard storage available at the satellite [1]. Hence, it is necessary to adopt the process of combining a high spatial and spectral resolution into one image is also called as Pansharpening [2]. This process generates a high-resolution MS image which greatly describes earth images. Important factor to be considered is fused image should contain more interpretation capability, also addition of spatial information without distracting spectral information [3].
The widely used pansharpening techniques are sorted into two key categories: component substitution pansharpening technique (CS) [4]. Comprehensive study of various image fusion algorithm along with applications are available in the literature [5]- [11]. The earlier is based upon on the exchange of element with PAN image. In this, spectral transformation of the MS image is worked as a component. Intensity hue saturation (IHS) [12]- [15], Gram Schmidt (GS) [16] and principal component analysis [17]- [20] are extensively used component substitution pansharpening techniques. HIS pan-sharpening technique works as following steps: first it converts given low resolution multispectral images into IHS model then the extracted intensity I is exchanged by high-resolution panchromatic image. And finally, inverse transform helps to generate high-resolution multispectral image. But the merged image generated using this technique always deficit due to spectral distortion even if contains high spatial resolution [21].
The latter approach is based on the partition of the images and then reconstruction of new image with spatial and spectral details. The several categories of MRA pan-sharpening techniques are decimated wavelet transform (DWT) [22], undecimated wavelet transform (UDWT) [23], "à trous" wavelet transform (ATWT) [24] and Laplacian pyramid (LP) [25]. In Wavelet transform (WT) pan-sharpening technique, WT is applied on both images and decomposed into two sub-images: approximation image which includes generalization of the source images and detail image comprises of detail description of the source images. Then, the approximate and the detail sub-images are combined to generate a merged image. The resultant image obtained through inverse WT. This category of pansharpening technique preserves spectral information but with some spatial distortions like blurring [26]. Hybrid approach for image fusion involves integration of CS and MRA are investigated in [27]- [31]. In this paper, the proposed hybrid algorithm is described as, initially, IHS transform is applied on intensity component of MS image.
SFIM and wavelet transform is applied on a result of the above step and PAN image to generate outcome with multiscale representation. Spectral property of the resultant image is preserved using SFIM. Information accommodated in approximate image is distinct from detail sub-image so those need to be processed separately. Finally, the inverse wavelet transforms (IWT) as well as an inverse intensity hue saturation (IHS) transforms are implemented to get the resultant fusion image. The superiority of the hybrid algorithm is proved through the empirical outcomes obtained with the help of Worldview-II images. This paper is organized into different modules such as Module II defines the traditional image fusion algorithms. Module III represents the proposed hybrid pan-sharpening algorithm. The experimental results with analysis for images of Worldview-II dataset are showcased in module IV. At last, conclusion is stated in module V.

Image fusion using IHS
IHS is one of the component substitution technique, where given an image, is transformed into IHS from RGB and I component is recovered by given PAN image and at the end resultant image is gained through inverse IHS transform as described by the given algorithm below.

Image fusion using SFIM
A proportion of a MS and PAN image and it's a smoothing filtered image used in SFIM helps to improve spatial resolution for spectral bands.

Image fusion using wavelet Transform (WT)
Wavelet transform is more attractive technique of image fusion due to its multiscale characteristics which are best suited to accomplish different image resolutions. These are classified into discrete and continuous categories [13]- [17], [22]. Here it considers a discrete approach.

THE PROPOSED METHOD
This research discusses about the hybrid image fusion algorithm using integration of CS and MRA algorithms as CS pansharpening technique encompasses the ability to improve spatial resolution while MRA pansharpening technique incorporates the ability to extract adequate spatial details through preserving spectral details. The proposed algorithm uses separate fusion rule criterion for approximate and detail subimage. The detail sub-image comprises of the exact description about the image, therefore, the maximum local extrema are selected to merge detail sub-images obtained from wavelet transform. The proposed algorithm is described to

RESULTS AND ANALYSIS
In this paper, dataset generated by a high-resolution WorldView-II satellite is used to verify the effectivity of the hybrid algorithm. This satellite provides multispectral and panchromatic images with 1.84-m and 0.46-m resolution correspondingly. The set of images used are available on the website http://www.datatang.com/data/43234. This dataset is created using Digital Globe and organized by Beijing key laboratory of digital media, Beihang University [21]. The images of this dataset describe the area of city, coastal, bridge, etc.

Evaluation indicators
This paper uses quality metrics like correlation coefficient, signal to noise ratio, peak signal to noise ratio, structural similarity, the root means squared error and percentage fit error for comparison of original and resultant images [34]- [38]. Table 1 shows in definition of the variable used in the quality metrics.  [37], [38] can be used to measure a presence of spectral information.
The resultant value of equation reveals the correlativity of fused and MS image and CC is defined as: b. The definition of the peak signal to noise ratio (PSNR) [42], [43] is represented a: Higher value of PSNR indicates noise in the merged image. c. The structural similarity (SSIM) [39], [42], [43] is defined as: SSIM is used to find similarity among MS and fused image; More the SSIM value then the chance to get similar images is also more. d. The root mean squared error (RMSE) [37], [38]

Experimental results
This paper also considers execution of other image fusion techniques like IHS [12]- [15], DWT [24]- [26], [39]- [42], BT [43] and quality measures like RMSE, PFE, MAE, CORR, SNR, PSNR, QI and SSIM for two different groups of images where each group contains 2 pairs of input images. This experiment and implementation is conducted in MATLAB2011a on a Windows8 computer.
Experimental results shown in Tables 2-5 supports that the proposed method provides an improved result than the other techniques like IHS, BT, and DWT. The highlighted experimental results indicate the superiority of the technique in an evaluation. It also indicates that the hybrid method improves the result with RMSE, PFE, MAE, SNR, PSNR, and SSIM for both sets of images but not able to improve the result for some images in terms of CORR present in Table 4.
We also observed that our proposed method generates a resultant image with high spectral and spatial resolution from human eye's visual perception as compared to others. The value of RMSE, PFE, SNR, PSNR, and SSIM specified in Tables 2-5 verifies that the proposed algorithm works best for the given set of images over the other pansharpening algorithms. It is observed that DWT slight edge over the proposed algorithm for Worldview-II seaside images of set 1 on correlation parameters, shown in Table 4.  Figures 1-4 (ii). The resultant fused image obtained by traditional fusion methods like BT, IHS, DWT and the proposed method are presented in Figures 1-4 (iii), (iv), (v) and (vi). respectively. Resultant fused image after applying proposed algorithm improves the clarity of the image with more accurate geographical details are observed in Figures 1-4 (vi).

CONCLUSION
The research work is focused on implementation of a hybrid image fusion algorithm which merges CS approach and MRA approach to overcome drawbacks of individual approach. This hybrid approach is designed to enhance superiority of images. Separate fusion rules are adopted in this hybrid approach for the approximate and the detail sub-images. The research work also involves comparison of hybrid pansharpening technique with some traditional algorithm like IHS, DWT, BT. Experiments are conducted on 4 sets of images from Worldview-II. The Superiority of a resultant merged image is calculated in terms of RMSE, PFE, MAE, CORR, SNR, PSNR, and SSIM as shown in Table 2-5. Empirical outcomes demonstrate that the hybrid algorithm gives improved fusion result than the others. The fused image obtained from the hybrid algorithm progresses best in resolution, so it can be used further for image processing applications for classification, feature extraction etc.
Current work can be altered in future through implementing different fusion rules for merging approximate and detail sub-images. Also, this work can be extended by developing a different hybrid approach for satellite images generated from different satellites.