Image Fusion Based On Extensions of Independent Component Analysis
Image Fusion Based On Extensions of Independent Component Analysis
Image Fusion Based On Extensions of Independent Component Analysis
ANALYSIS
KEY WORDS: image processing, integration, image understanding, fusion, land cover
ABSTRACT:
Remote sensing image fusion can effectively improve the accuracy of image classification. This paper proposes an image fusion
algorithm based on extensions of independent component analysis (ICA) and multi-classifier system. Firstly a novel method of
fusing panchromatic and multi-spectral remote sensing images is developed by contourlet transform which can offer a much richer
set of directions and shapes than wavelet. As ICA not only can effectively remove the correlation of multi-spectral images, but also
can realize sparse coding of images and capture the essential edge structures and textures of images, then using features extracted
from the extension of ICA domain coefficients of the fused image, different classifiers corresponding to different image features are
chosen in parallel style and the support vector machines are trained to classify the whole fused image as stack part in the proposed
multi-classifier system. Experimental results show that the proposed algorithm can effectively improve the accuracy of image
classification.
This paper proposes an image fusion algorithm of remote Recently developed contourlet transform can offer a much
sensing images based on extensions of independent component richer set of directions and shapes, and thus it is more effective
analysis (ICA) and multi-classifier system. Firstly a novel than wavelet in capturing smooth contours and geometric
method of fusing panchromatic and multi-spectral remote structures in images. This paper proposes a novel method of
sensing images is developed by contourlet transform. Then fusing panchromatic and multi-spectral remote sensing images
using different features extracted from the extension of ICA based on contourlet transform.
domain coefficients of the fused images, a parallel and stack
multi-feature and multi-classifier decision level image fusion
* Corresponding author.
1111
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
1112
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
3.2.3 Modified Learning Rule: To resolve the separation A = (a1 , a2 ,L, a N ) ,column vector ai (i = 1,2,L, N ) denotes
matrix W, the optimization problem can be induced as follows:
a group of N × 1 pixels basis images. Through ICA resolves
2
i =1
Where η is learning rate, here the self-adaptive adjustment Figure 1. ICA basis of natural image data
method is developed in this paper.
1113
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
⎧I1 = (R + G + B) / 3
⎪
⎨I 2 = (R − B) / 2 (14)
4. IMAGE FUSION ALGORITHM BASED ON ⎪I = (2G − R − B) / 4
EXTENSIONS OF ICA AND MULTI-CLASSIFIER
⎩3
SYSTEM
This part introduces the decision level remote sensing image The components in Ohta color space are irrelevant, so it can
fusion algorithm. Existing studies have show that by fusing well apperceive the change of color in statistical view. Ohta
multi-spectral images and panchromatic image to get the high color space is gotten by linear transform of RGB color space,
quality fusion images can improve the accuracy of classification here I1 is the intensity component, I2 and I3 are the almost
than just using single source image. This paper uses the orthogonal color components. The feature vectors obtained
panchromatic image, multi-spectral images and the resulting from Ohta color space are denoted by T1 here.
fused images for researching objects, extracting the spectral
features, texture features and the features in ICA and TICA 4.2.2 Texture Feature of TICA basis: Since the original
transformation domain, using different classifiers to get the panchromatic image has high spatial resolution and low spectral
classification results corresponding to different features and resolution, different objects have the same gray value or the
applying the method of multi-classifier system to obtain the same object has different gray values in the panchromatic
final classification of land cover from remote sensing images. image sometimes. As the fused images can have high spatial
and spectral resolution simultaneously, the fused color images
4.1 Image Fusion Based on Contourlet Transform are transformed into the grayscale image named grayscale
modulation image (GMI) here by specific algorithm. GMI is
Coregister both images and resample the multi-spectral images useful information source because its spatial resolution is
to make its pixel size equal to that of the panchromatic image in similar to the original panchromatic image and its gray spectral
order to get perfectly superposable images. Here only R/G/B values can reflect different objects much better. So by applying
three channels are considered. The images are firstly TICA to the GMI block by block in sliding window style, the
decomposed by contourlet transform, getting low frequency and texture features of TICA basis can be extracted.
high frequency coefficients in different resolutions and different
directions. Then combining ~ a trous wavelet, the fusion In order to uncover the underlying structure of an image, it is
procedure is choosing different rules on particular sets of common practice in image analysis to express an image as the
contourlet coefficients that correspond to high and low synthesis of several other basis images. These bases are chosen
frequency bands. The high-frequency coefficients of the according to serve some specific analysis tasks. The advantage
panchromatic image substitute all the high-frequency of the TICA basis is that the estimated transform can be tailored
coefficients in R/G/B three channels. The panchromatic image to the needs of the application. A set of images with similar
is decomposed by ~ a trous wavelet , getting a group of wavelet content to the GMI is selected for training the desired bases.
plane coefficients, then adding these wavelet coefficients to the Then using the TICA basis the GMI is transformed into the
low-frequency contourlet coefficients, that is further extracts TICA domain in sliding N × N window style. All of these
the detail information of panchromatic image for fusing extracted TICA features can well reflect the structure and
application. Final fused images are obtained by using reversed texture information of the fused images. The TICA feature
contourlet transform. vectors are denoted by T2 here.
The proposed method can get more information in the fused 4.2.3 Independent Component Feature: The existing
results and the spectral reserving character is quite well. The studies show that the correlation between the bands of multi-
contourlet transform image fusion method offers a desirable spectral images sometimes brings ill effect in image
result to improve spatial resolution and information of the fused classification. ICA not only can remove the correlation in the
images, which get ready for the next classification procedure. bands of multi-spectral images, but also can makes the resulting
components mutual independent as much as possible. The every
4.2 Image Feature Extraction resulting band of independent component embodies a
concentrated reflection of certain ground objects, increasing the
4.2.1 Spectral Feature: The spectral features of multi- degree of separation between different ground objects.
spectral images are the most essential features, here the three Therefore independent component analysis can effectively
channels of high spatial resolution fused images are considered. remove the unfavorable influence and raise the accuracy of
Besides the spectral features extracted in the original R/G/B classification.
color space, other image features can be gotten by transforming
the multi-spectral fused image into different color space. How Multi-spectral images can be regarded as the linear combination
to choose the suitable color space is an important factor for of multi-source mixture signals in some sense due to their low
different color space can define different useful spectral spatial resolution. This paper treats the panchromatic and multi-
features. So there are two key points should be considered in spectral images as four dimensional mixture signals and adopts
choosing color space, one is that the color space can present the ICA to obtain another fusion scheme of multi-spectral and
irrelevant color features, the other is that the color space panchromatic images to get three dimensional multi-spectral
remains constant in different illumination conditions. Based on images. Though the resulting independent components can not
the above considerations, the paper chooses the Ohta color reserve the original spectral characteristics very well, they can
space(Ohta,1985) which can be expressed as following. express the sharpened multi-spectral images and resolve the
problem of unmixing the mixed multi-spectral images pixels in
the hypothesis of linear spectral mixture model, which lay the
foundation for the next step of properly classification. The
1114
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
extracted three independent component features are recorded as hidden units from the input layer and sent to the output layer,
T3 here. neural state of each layer only affects state of the next layer. If
the expected output can not be obtained in the output layer, so
By means of the methods of feature extraction mentioned above, transfer to back propagation, and let error signal back along the
this paper can get three major feature of fused images, i.e. original link pathway, the error signal can became least through
spectral feature (T1), texture feature of TICA (T2) and linear amend the values of each nerve cell. This paper chooses the BP
transform feature of ICA (T3). neural network with one hidden layer and uses C2 denotes it.
4.3 Multi-Classifier Construction 3. Decision tree classifier. The decision tree classifier is a set of
hierarchical rules which are successively applied to the input
4.3.1 Principle of Multi-Classifier System: Classification data. Those rules are thresholds used to binary split the data into
is the process of assigning presented information into classes two groups. Each node is such that the descendant nodes are
and categories of the same type. The classification of the image purer in terms of classes. Decision tree rules are explicit and
requires the estimation of the posterior probability for each allow for identification of features which are relevant to
class. Such estimates can be obtained by using supervised and distinguish specific classes. Then the analysis is reduced to the
unsupervised classification algorithms. most useful layers. The structure of the decision tree can also be
reveal hierarchical and nonlinear relationships among input
The output of a classifier can take abstract form, rank level and layers. These relationships often result in a given class being
measurement level. In the past few years, significant efforts described by various terminal nodes. Terminal nodes are the
have been devoted to the development of effective algorithms final decision, which assign a sample to certain class. Here
for combining different types of classifiers in order to exploit decision tree classifier is denoted by C3.
the complementary information that they
provide(Burzzone,2001; Ranawana,2006). So if a multi- 4. Support vector machines, SVMs. Support vector machines
classifier system is to be successful, the different classification (SVMs) is a kind of machine learning based on statistical
should have good individual performances and be sufficiently learning theory(Vladmir,2000). The basic idea of applying
different from each other. A multi-classifier can be constructed SVMs to pattern classification can be stated briefly as follows:
either in a parallel, stack or combined manner. Once the firstly map the input vectors into one feature space, either
individual classifiers have been designed and implemented, the linearly or non-linearly, which is relevant with the selection of
next most important task involves the combination of the the kernel function. Then with the feature space from the first
individual results obtained through each individual classifier. step construct a hyperplane which separates two classes,.This
The strategy includes linear combination methods, non-linear can be extended to multi-class.
combination methods, statistical methods and computationally
intelligent method. The commonly used four kernel function in SVMs are: linear
function, polynomial function, radial basis function, sigmoid
The success of a multi-classifier system depends on three key function. SVMs have the important computational advantage
features: proper selection of classifier with diversity, topology that no nonconvex optimization is involved. Moreover, its
and combinational methodology. The main purpose of multi- performance is related to the margin with which it separates the
classifier combination is to take advantage of the different data. As a new classification technique, SVMs outperforms
classifiers to enhance the generalization ability of the individual many conventional approaches in various applications. Here
classifier to gain the better results of classification. This paper SVMs classifier is denoted by C4.
makes a useful attempt in the multi-classifier system and
proposes a multi-classifier fusion method base on extension of 4.3.3 Strategy of Multi-Classifier Fusion: Corresponding
ICA. to the three different features extracted from the fused images
and the four different selected classifiers, this paper constructs
4.3.2 Classifier Selection: Corresponding to the three the parallel topology of multi-classifiers firstly, detail
different features in the fused images, this paper makes the descriptions are as followings.
pointed choice the following classifiers, including K-NN
classifier, BP neural network classifier, decision tree classifier Towards the spectral features T1 in Ohta color space, K-NN C1
and multi-category SVMs. and decision tree C3 are chosen and combined in parallel
topology. All the feature vectors are put into the two classifiers
1. K-nearest neighbor classifier, K-NN. The K-NN has a very and respective classification results are obtained in parallel
effective strategy as a learner, it keeps all training instances. A topology style.
classification is made by measuring the distances from the test
instance to all training instances, most commonly using the For texture features of TICA basis, the paper chooses K-NN C1
Euclidean distance. From these distances, a distance matrix is and BP neural network classifier C2 and combines them in
constructed between all possible pairings of points. The data parallel style, resulting two respective classification results.
points, k-closest neighbors are then found by analyzing the In regard to independent component features T3, K-NN C1 is
distance matrix. The k-closest data points are then analyzed to chosen to get the corresponding classification results.
determine which class label is the most common among the set.
Finally the majority class among the K nearest instances is A number of training area for different classes are chosen in the
assigned to the test instance. K-NN classifier is denoted by C1 study images, following the above methods to extract spectral
here. and ICA/TICA image features, then training all the chosen
classifiers and the trained classifiers are applying to classify the
2. BP neural network classifier. Back-propagating network (BP whole fused images every pixel. Through different classifiers
network) is a type of neural network. When positive direction the corresponding posterior probability of different
spread, the imported model disposes layer by layer by way of
1115
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
classification results are gotten, so decision level fusion of these values in Ohta color space correspond to the first type of input
classification results is needed. feature vectors, i.e. T1.
4.3.4 Decision Fusion Strategy: There are many strategies The fused color images are transformed into the GMI. Then
for combining classification results of each individual classifier, applying TICA to the GMI with 2×2 pixels of block in sliding
of which majority voting principle and Bayesian combination window style to get four coefficients of TICA domain,
strategy are the most common used fusion method. meanwhile, the statistical parameters, such as mean, standard
deviation, average gradient are computed for the second type of
1.Majority Voting Principle: The majority voting method input feature vectors, i.e. T2.
selects the relevant class by polling all the classifiers to see
which class is the most popular. Whichever class gets the Turning the original multi-spectral images into 3×65536 pixels
highest vote is selected. This method is particularly successful vectors and converting the original panchromatic image into
when the classifiers involved output binary votes. 1×65536 pixels vector to form 4×65536 pixels input vectors,
applying ICA to the whole input vectors, three independent
2.Bayesian Combination Strategy: Bayesian combiners are used component bands are shown in Figure.5 (a)~ (c). The results
to carried out the classification according to the Bayes rule by indicate that the three independent components play good role
selecting the class associated with the maximum average in separate the water body, naked land and dry land. Meanwhile
probability. the resulting fused false color images in Figure.5(d) have higher
spatial resolution compare to the original multi-spectral images.
3.The proposed Fusion Strategy: Different from the routine All these three independent components are chosen as feature
fusion strategy, this paper adopts SVMs C4 to fuse the different vectors for classification, i.e. T3.
classification results corresponding to different image features
to get the final fusion decision. Each classification results of The training areas for different classes are chosen in the images,
respective classifiers serve as the input feature vectors for following the above methods to extract spectral and ICA/TICA
training SVMs, which can be regarded as stack multi-classifier image features and training all the chosen classifiers, the trained
fusion style and the continuation of the aforementioned parallel classifiers are then applying to classify the whole fused images
multi-classifier system. The total fusion topology is as figure 3. every pixel. Selecting suitable SVMs kernel function and
Moreover the above common fusion rules are also used to get parameter to train multi-category SVMs with the input feature
the classification results corresponding to different features. vectors of every category obtaining from the afore parallel
classifiers. The trained multi-category SVMs are applying to
classify the whole fused images to gain the classification results.
Spectra This paper chooses the radial basis kernel function:
K-NN
feature (C1)
( T1)
Panchromatic,multi-
spectral and fused
2
K ( x, x' ) =exp( x - x' /2σ 2 ) ,where σ =2, c =100。
images
The training area of water body, naked land and dry land are
selected in the images, every category has two block of 16×16
pixels training samples, theses samples of three component
1116
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
Figure.4 Original SPOT(a), TM5/4/3 images(b) and fused images by contourle transform(c)
Waterbody
Naked land
Dry land
Figure.5 The three independent component bands(a,b,c) of the fused image by ICA(d)
Classification algorithm Total accuracy(%) As ICA not only can effectively remove the correlation of
Min Distance 62.58 multi-spectral images, but also can realize sparse coding of
images and capture the essential edge structures and textures of
Max Likelihood 65.44 images, then using features extracted from the extensions of
Majority Voting Principle 78.53 ICA domain coefficients of the images, different classifiers
corresponding to different features are chosen in parallel multi-
Bayesian Combination Strategy 81.80 classifier style and the SVMs as stack fusion style are trained
Proposed Algorithm 82.21 to classify the whole images in the proposed multi-feature and
multi-classifier system. Experimental results show that the
proposed algorithm can effectively improve the accuracy of
Table 1 the comparison of total accuracy of classification using image classification.
different methods
REFERENCE
6. CONCLUSION Thierry Ranchin, Bruno Aiazzi, Luciano Alparone, Stefano
Baronti, Lucien Wald,2003. Image fusion—the ARSIS concept
Remote sensing image classification is an important means for and some successful implementation schemes, ISPRS Journal of
quantified remote sensing image analysis, and remote sensing Photogrammetry & Remote Sensing, 58 :4~18
image fusion can effectively improve the accuracy of image
classification. This paper proposes an image fusion algorithm M.N.Do,M.Vetterli,2002 .Contourlets: A Directional
based on extension of ICA and multi-classifier system. A novel Multiresolution Image Representation, IEEE ICIP:357~360
method of fusing panchromatic and multi-spectral remote
sensing images is developed by contourlet transform that can Aapo Hyvarinen,1999. Survey on independent component
offer a much richer set of directions and shapes than wavelet. analysis, Neural Computing Surveys, 2:94~128
1117
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B7. Beijing 2008
1118