Ensemble Methods for Classification of Hyperspectral Data

Lecture Notes in Computer Science, 2009

The classification of hyperspectral imagery, using multiple classifier systems is discussed and an SVM-based ensemble is introduced. The data set is separated into separate feature subsets using the correlation between the different spectral bands as a criterion. Afterwards, each source is classified separately by an SVM classifier. Finally, the different outputs are used as inputs for final decision fusion that is based on an additional SVM classifier. The results using the proposed strategy are compared to classification results achieved by a single SVM and other well known classifier ensembles, such as random forests, boosting and bagging.

The improved spectral resolution of modern hyperspectral sensors provides capabilities for discrimination of subtly different classes and objects. However, in order to obtain statistically reliable classification results, the number of required training samples increases exponentially as the number of spectral bands increases. However, in many situations, acquisition of the large number of training samples for the high-dimensional datasets may not be feasible. Multiple classifiers have been regarded as a promising solution for this problem. In this paper, creation of ensemble of classifiers based on feature selection has been evaluated and an effective strategy for generation of feature subsets has been proposed. The proposed method is based on generating multiple feature subsets by running feature selection algorithm several times, with the aim of discrimination of one class from the others each time. Each of the final subsets of features is selected so as to have the capability for discrimination of one of the classes. Each of these subsets is then passed to the maximum likelihood classifier. Finally a combination scheme is used to combine the outputs of individual classifiers. Practical examinations on the AVIRIS data for discrimination of different land cover classes demonstrate the effectiveness of the proposed strategy.

Proceedings 5th EARSeL …, 2007

Machine learning algorithms are methods developed to deal with large volumes of data with high efficiency. Adaboost has been among the most popular and promising algorithms in the last decade and has demonstrated its potential for classification of remote sensing data. Previous studies have shown that Adaboost, though less stable than bagging (another well-know ensemble classification algorithm), consistently produces higher accuracies in classification tasks performed in a vast variety of data domains. The use of Adaboost for hyperspectral classification, however, has not been fully explored. Like Adaboost, Random Forest is another bootstrap method proposed recently to generate numerous, up to hundreds of classifiers for classification. Using the same resampling strategy as bagging, Random Forest introduces a new feature, called out-of-bag samples, for feature ranking and evaluation. The only parameter for tuning is the number of features to split on at each node, which is described as insensitive to accuracy. Comparatively, Adaboost does not have any parameters except for the amount of pruning, which is zero when using Random Forest. In this paper, we compare the results obtained with both classifiers on hyperspectral data. Results from two applications, one on ecotope mapping and one on urban mapping are presented. Compared with using one decision tree classifier, Adaboost increases classification accuracy by 9%, and Random Forest by 13%. Both classifiers achieve comparable results in terms of overall accuracy. Random Forest, however, due to its use of only a random feature subset and no pruning, is more efficient. Our results show that both Adaboost and Random Forest are exceptionally fast in training and achieve higher accuracies than accurate classifiers such as Multi-Layer Perceptrons. Their limited demands on user's input for parameter tuning makes them ideal algorithms for operationally oriented tasks. The study demonstrates that Adaboost and Random Forest perform well with hyperspectral data, in terms of both accuracy and ease-of-use.

IEEE Geoscience and Remote Sensing Letters, 2000

In real applications, it is difficult to obtain a sufficient number of training samples in supervised classification of hyperspectral remote sensing images. Furthermore, the training samples may not represent the real distribution of the whole space. To attack these problems, an ensemble algorithm which combines generative (mixture of Gaussians) and discriminative (support cluster machine) models for classification is proposed. Experimental results carried out on hyperspectral data set collected by the reflective optics system imaging spectrometer sensor, validates the effectiveness of the proposed approach.

International Journal of Image and Data Fusion, 2010

Classification of hyperspectral data using a classifier ensemble that is based on support vector machines (SVMs) are addressed. First, the hyperspectral data set is decomposed into a few data sources according to the similarity of the spectral bands. Then, each source is processed separately by performing classification based on SVM. Finally, all outputs are used as input for final decision fusion performed by an additional SVM classifier. Results of the experiments underline how the proposed SVM fusion ensemble outperforms a standard SVM classifier in terms of overall and class accuracies, the improvement being irrespective of the size of the training sample set. The definition of the data sources resulting from the original data set is also studied.

2014

Image classification is one of the most important tasks of remote sensing information processing used for object recognition. In this paper, a novel scheme is proposed to improve the accuracy of hyperspectral image classification by amalgamating multiple feature vector sets and ensemble methods with different classifiers. Extracting the texture, color and object features of the satellite images, an ensemble classifier is built for object recognition which recognizes the type of objects present in it. Effective use of feature set and the selection of suitable classification methods with different combination methods are applied for improving classification accuracy. Classifiers such as Multi Layer Perceptron (MLP), k-Nearest Neighbour (KNN) and Support Vector Machine (SVM) are used. This combination shows high performance in terms of Classifier Accuracy (CA), Object Recognition Rate (ORR) and False Alarm Rate (FAR). Results obtained from the ensembling classification give better solu...

Journal of the Indian Society of Remote Sensing, 2013

One of the most widely used outputs of remote sensing technology is Hyperspectral image. This large amount of information can increase classification accuracy. But at the same time, conventional classification techniques are facing the problem of statistical estimation in high-dimensional space. Recently in remote sensing, support vector machines (SVMs) have shown very suitable performance in classifying high dimensionality problem. Another strategy that has recently been used in remote sensing is multiple classifier system (MCS). It can also improve classification accuracy by combining different classifier methods or by a diversity of the same classifier. This paper aims to classify a Hyperspectral data using the most common methods of multiple classifier systems i.e. adaboost and bagging and a MCS based on SVM. The data used in the paper is an AVIRIS data with 224 spectral bands. The final results show the high capability of SVMs and MCSs in classifying high dimensionality data.

Hyperspectral Data Exploitation, 2007

In the recent years, pixel-wise classification of hyperspectral images aroused many developments, and the literature now provides various classifiers for numerous applications. In this chapter, we present a generic framework where the redundant or complementary results provided by multiple classifiers can actually be aggregated. Taking advantage from the specificities of each classifier, the decision fusion thus increases the overall classification performances. The proposed fusion approach is in two steps. In a first step, data are processed by each classifier separately and the algorithms provide for each pixel membership degrees for the considered classes. Then, in a second step, a fuzzy decision rule is used to aggregate the results provided by the algorithms according to the classifiers' capabilities. The general framework proposed for combining information from several individual classifiers in multiclass classification is based on the definition of two measures of accuracy. The first one is a point-wise measure which estimates for each pixel the reliability of the information provided by each classifier. By modeling the output of a classifier as a fuzzy set, this point-wise reliability is defined as the degree of uncertainty of the fuzzy set. The second measure estimates the global accuracy of each classifier. It is defined a priori by the user. Finally, the results are aggregated with an adaptive fuzzy fusion ruled by these two accuracy measures. The method is illustrated by considering the classification of hyperspectral remote sensing images from urban areas. It is tested and validated with two classifiers on a ROSIS image from Pavia, Italy. The proposed method improves the classification results when compared with the separate use of the different classifiers. The approach is also compared to several other standard fuzzy fusion schemes.

Pattern Recognition Letters, 2016

Due to the high-dimension characteristics of hyperspectral data, dimensionality reduction is becoming an important problem in hyperspectral image classification. Band selection can retain the information which is capable of keeping the original physical meaning of the spectral channels, and thus it has attracted more research interests. This paper tackles the band selection problem from the perspective of multiple classifiers combination, which can obtain higher classification accuracy. In the newly formulated framework of band selection and classification based on combination of multiple classifiers (BS_CMC), stochastic algorithms are firstly employed to generate several groups of initial band subset, on which a pool of classifiers is constructed. Then, improved classifier selection algorithm based on error diversity is proposed to select several member classifiers from the initial classifier pool. And finally the classification is performed through dynamic classifier selection based on local classification accuracy. The experimental results on two benchmark data sets show that the proposed approach can select those bands with more discriminative information and improve the classification accuracy effectively.

Pattern Analysis & Applications, 2002

Many classification problems involve high dimensional inputs and a large number of classes. Multiclassifier fusion approaches to such difficult problems typically centre around smart feature extraction, input resampling methods, or input space partitioning to exploit modular learning. In this paper, we investigate how partitioning of the output space (i.e. the set of class labels) can be exploited in a multiclassifier fusion framework to simplify such problems and to yield better solutions. Specifically, we introduce a hierarchical technique to recursively decompose a C-class problem into CϪ1 two-(meta) class problems. A generalised modular learning framework is used to partition a set of classes into two disjoint groups called meta-classes. The coupled problems of finding a good partition and of searching for a linear feature extractor that best discriminates the resulting two meta-classes are solved simultaneously at each stage of the recursive algorithm. This results in a binary tree whose leaf nodes represent the original C classes. The proposed hierarchical multiclassifier framework is particularly effective for difficult classification problems involving a moderately large number of classes. The proposed method is illustrated on a problem related to classification of landcover using hyperspectral data: a 12-class AVIRIS subset with 180 bands. For this problem, the classification accuracies obtained were superior to most other techniques developed for hyperspectral classification. Moreover, the class hierarchies that were automatically discovered conformed very well with human domain experts' opinions, which demonstrates the potential of using such a modular learning approach for discovering domain knowledge automatically from data.