Decision Fusion

Visual speech information from the speaker's mouth region has been successfully shown to improve noise robustness of automatic speech recognizers, thus promising to extend their usability into the human computer interface. In this paper, we review the main components of audio-visual automatic speech recognition and present novel contributions in two main areas: First, the visual front end design, based on a cascade of linear image transforms of an appropriate video region-of-interest, and subsequently, audio-visual speech integration. On the later topic, we discuss new work on feature and decision fusion combination, the modeling of audio-visual speech asynchrony, and incorporating modality reliability estimates to the bimodal recognition process. We also briefly touch upon the issue of audiovisual speaker adaptation. We apply our algorithms to three multi-subject bimodal databases, ranging from small-to largevocabulary recognition tasks, recorded at both visually controlled and challenging environments. Our experiments demonstrate that the visual modality improves automatic speech recognition over all conditions and data considered, however less so for visually challenging environments and large vocabulary tasks.

This paper evaluates strategies for atlas selection in atlas-based segmentation of three-dimensional biomedical images. Segmentation by intensity-based nonrigid registration to atlas images is applied to confocal microscopy images acquired from the brains of 20 bees. This paper evaluates and compares four different approaches for atlas image selection: registration to an individual atlas image (IND), registration to an average-shape atlas image (AVG), registration to the most similar image from a database of individual atlas images (SIM), and registration to all images from a database of individual atlas images with subsequent multi-classifier decision fusion (MUL). The MUL strategy is a novel application of multi-classifier techniques, which are common in pattern recognition, to atlas-based segmentation. For each atlas selection strategy, the segmentation performance of the algorithm was quantified by the similarity index (SI) between the automatic segmentation result and a manually generated gold standard. The best segmentation accuracy was achieved using the MUL paradigm, which resulted in a mean similarity index value between manual and automatic segmentation of 0.86 (AVG, 0.84; SIM, 0.82; IND, 0.81). The superiority of the MUL strategy over the other three methods is statistically significant (two-sided paired t test, P < 0.001). Both the MUL and AVG strategies performed better than the best possible SIM and IND strategies with optimal a posteriori atlas selection (mean similarity index for optimal SIM, 0.83; for optimal IND, 0.81). Our findings show that atlas selection is an important issue in atlas-based segmentation and that, in particular, multi-classifier techniques can substantially increase the segmentation accuracy.

Information fusion by utilizing multiple distributed sensors is studied in this work. Extending the classical parallel fusion structure by incorporating the fading channel layer that is omnipresent in wireless sensor networks, we derive the likelihood ratio based fusion rule given fixed local decision devices. This optimum fusion rule, however, requires perfect knowledge of the local decision performance indices as well as the fading channel. To address this issue, two alternative fusion schemes, namely, the maximum ratio combining statistic and a two-stage approach using the Chair-Varshney fusion rule, are proposed that alleviate these requirements and are shown to be the low and high signal-to-noise ratio (SNR) equivalents of the likelihood-based fusion rule. To further robustify the fusion rule and motivated by the maximum ratio combining statistics, we also propose a statistic analogous to an equal gain combiner that requires minimum a priori information. Performance evaluation is performed both analytically and through simulation.

The classification of multisensor data sets, consisting of multitemporal synthetic aperture radar data and optical imagery, is addressed. The concept is based on the decision fusion of different outputs. Each data source is treated separately and classified by a support vector machine (SVM). Instead of fusing the final classification outputs (i.e., land cover classes), the original outputs of each SVM discriminant function are used in the subsequent fusion process. This fusion is performed by another SVM, which is trained on the a priori outputs. In addition, two voting schemes are applied to create the final classification results. The results are compared with well-known parametric and nonparametric classifier methods, i.e., decision trees, the maximumlikelihood classifier, and classifier ensembles. The proposed SVM-based fusion approach outperforms all other approaches and significantly improves the results of a single SVM, which is trained on the whole multisensor data set.

In this contribution we introduce a novel approach to the combination of acoustic features and language information for a most robust automatic recognition of a speaker's emotion. Seven discrete emotional states are classified throughout the work. Firstly a model for the recognition of emotion by acoustic features is presented. The derived features of the signal-, pitch-, energy, and spectral contours are ranked by their quantitative contribution to the estimation of an emotion. Several different classification methods including linear classifiers, Gaussian Mixture Models, Neural Nets, and Support Vector Machines are compared by their performance within this task. Secondly an approach to emotion recognition by the spoken content is introduced applying Belief Network based spotting for emotional key-phrases. Finally the two information sources will be integrated in a soft decision fusion by using a Neural Net. The gain will be evaluated and compared to other advances. Two emotional speech corpora used for training and evaluation are described in detail and the results achieved applying the propagated novel advance to speaker emotion recognition are presented and discussed.

A novel atlas-based segmentation approach based on the combination of multiple registrations is presented. Multiple atlases are registered to a target image. To obtain a segmentation of the target, labels of the atlas images are propagated to it. The propagated labels are combined by spatially varying decision fusion weights. These weights are derived from local assessment of the registration success.

In this letter, a technique based on independent component analysis (ICA) and extended morphological attribute profiles (EAPs) is presented for the classification of hyperspectral images. The ICA maps the data into a subspace in which the components are as independent as possible. APs, which are extracted by using several attributes, are applied to each image associated with an extracted independent component, leading to a set of extended EAPs. Two approaches are presented for including the computed profiles in the analysis. The features extracted by the morphological processing are then classified with an SVM. The experiments carried out on two hyperspectral images proved the effectiveness of the proposed technique. Index Terms-Attribute filters, decision fusion, extended attribute profile (EAP), independent component analysis (ICA), mathematical morphology, remote sensing.

Three-dimensional atlases and databases of the brain at different ages facilitate the description of neuroanatomy and the monitoring of cerebral growth and development. Brain segmentation is challenging in young children due to structural differences compared to adults. We have developed a method, based on established algorithms, for automatic segmentation of young children's brains into 83 regions of interest (ROIs), and applied this to an exemplar group of 33 2-year-old subjects who had been born prematurely. The algorithm uses prior information from 30 normal adult brain magnetic resonance (MR) images, which had been manually segmented to create 30 atlases, each labeling 83 anatomical structures. Each of these adult atlases was registered to each 2-year-old target MR image using non-rigid registration based on free-form deformations. Label propagation from each adult atlas yielded a segmentation of each 2-year-old brain into 83 ROIs. The final segmentation was obtained by combination of the 30 propagated adult atlases using decision fusion, improving accuracy over individual propagations. We validated this algorithm by comparing the automatic approach with three representative manually segmented volumetric regions (the subcortical caudate nucleus, the neocortical precentral gyrus and the archicortical hippocampus) using similarity indices (SI), a measure of spatial overlap (intersection over average). SI results for automatic versus manual segmentations for these three structures were 0.90 ± 0.01, 0.90 ± 0.01 and 0.88 ± 0.03 respectively. This registration approach allows the rapid construction of automatically labelled age-specific brain atlases for children at the age of 2 years.

Multisource classification methods based on neural networks and statistical modeling are considered. For these methods, the individual data sources are at first treated separately and modeled by statistical methods. Then several decision fusion schemes are applied to combine the information from the individual data sources. These schemes include weighted consensus theory where the weights of the individual data sources reflect the reliability of the sources. The weights are optimized in order to improve the combined classification accuracies. Other considered decision fusion schemes are based on two-stage approaches which use voting in the first stage and reject samples if either the majority or all of the classifiers for the data sources do not agree on a classification of a sample. For the second stage, a neural network is used to classify the rejected samples. The proposed methods are applied in the classification of multisource and hyperdimensional data sets, and the results compared to accuracies obtained with conventional classification schemes.

The classification of very high-resolution remote sensing images from urban areas is addressed by considering the fusion of multiple classifiers which provide redundant or complementary results. The proposed fusion approach is in two steps. In a first step, data are processed by each classifier separately and the algorithms provide for each pixel membership degrees for the considered classes. Then, in a second step, a fuzzy decision rule is used to aggregate the results provided by the algorithms according to the classifiers' capabilities. In this paper, a general framework for combining information from several individual classifiers in multiclass classification is proposed. It is based on the definition of two measures of accuracy. The first one is a point-wise measure which estimates for each pixel the reliability of the information provided by each classifier. By modeling the output of a classifier as a fuzzy set, this point-wise reliability is defined as the degree of uncertainty of the fuzzy set. The second measure estimates the global accuracy of each classifier. It is defined a priori by the user. Finally, the results are aggregated with an adaptive fuzzy operator ruled by these two accuracy measures. The method is tested and validated with two classifiers on IKONOS images from urban areas. The proposed method improves the classification results when compared with the separate use of the different classifiers. The approach is also compared to several other fuzzy fusion schemes.

This paper presents four schemes for soft fusion of the outputs of multiple classi®ers. In the ®rst three approaches, the weights assigned to the classi®ers or groups of them are data dependent. The ®rst approach involves the calculation of fuzzy integrals. The second scheme performs weighted averaging with data-dependent weights. The third approach performs linear combination of the outputs of classi®ers via the BADD defuzzi®cation strategy. In the last scheme, the outputs of multiple classi®ers are combined using Zimmermann's compensatory operator. An empirical evaluation using widely accessible data sets substantiates the validity of the approaches with data-dependent weights, compared to various existing combination schemes of multiple classi®ers. Ó 0167-8655/99/$ ± see front matter Ó 1999 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 -8 6 5 5 ( 9 9 ) 0 0 0 1 2 -4

The security of computer networks plays a strategic role in modern computer systems. In order to enforce high protection levels against threats, a number of software tools have been currently developed. Intrusion Detection Systems aim at detecting intruders who elude "first line" protection. In this paper, a pattern recognition approach to network intrusion detection based on the fusion of multiple classifiers is proposed. Five decision fusion methods are assessed by experiments and their performances compared. The potentialities of classifier fusion for the development of effective intrusion detection systems are evaluated and discussed.

This paper describes a novel hand gesture recognition system that utilizes both multi-channel surface electromyogram (EMG) sensors and 3D accelerometer (ACC) to realize user-friendly interaction between human and computers. Signal segments of meaningful gestures are determined from the continuous EMG signal inputs. Multi-stream Hidden Markov Models consisting of EMG and ACC streams are utilized as decision fusion method to recognize hand gestures. This paper also presents a virtual Rubik's Cube game that is controlled by the hand gestures and is used for evaluating the performance of our hand gesture recognition system. For a set of 18 kinds of gestures, each trained with 10 repetitions, the average recognition accuracy was about 91.7% in real application. The proposed method facilitates intelligent and natural control based on gesture interaction.

The 2008 Data Fusion Contest organized by the IEEE Geoscience and Remote Sensing Data Fusion Technical Committee deals with the classification of high-resolution hyperspectral data from an urban area. Unlike in the previous issues of the contest, the goal was not only to identify the best algorithm but also to provide a collaborative effort: The decision fusion of the best individual algorithms was aiming at further improving the classification performances, and the best algorithms were ranked according to their relative contribution to the decision fusion. This paper presents the five awarded algorithms and the conclusions of the contest, stressing the importance of decision fusion, dimension reduction, and supervised classification methods, such as neural networks and support vector machines.

The 2009-2010 Data Fusion Contest organized by the Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society was focused on the detection of flooded areas using multi-temporal and multi-modal images. Both high spatial resolution optical and synthetic aperture radar data were provided. The goal was not only to identify the best algorithms (in terms of accuracy), but also to investigate the further improvement derived from decision fusion. This paper presents the four awarded algorithms and the conclusions of the contest, investigating both supervised and unsupervised methods and the use of multi-modal data for flood detection. Interestingly, a simple unsupervised change detection method provided similar accuracy as supervised approaches, and a digital elevation model-based predictive method yielded a comparable projected change detection map without using post-event data.

In this paper, we present some recent developments of Multiple Classifiers Systems (MCS) for remote sensing applications. Some standard MCS methods (boosting, bagging, consensus theory and random forests) are briefly described and applied to multisource data (satellite multispectral images, elevation, slope and aspect data) for landcover classification. In a second part, special attention is given to Support Vector Machines (SVM) based algorithms. In particular, the fusion of two classifiers using the spectral and the spatial information, respectively, is discussed in the frame of hyperspectral remote sensing for the classification of urban areas. In all the cases, MCS provide a significant improvement of the classification accuracies. In order to address new challenges for the analysis of remote sensing data, MCS provide invaluable tools to handle situations with an ever growing complexity. Examples include extraction of multiple features from one data set, use of multi-sensor data, and complementary use of several algorithms in a decision fusion scheme.

This paper describes a multimodal approach for speaker veri cation. The system consists o f t wo c l a ssi ers, one using visual features and the other using acoustic features. A lip tracker is used to extract visual information from the speaking face which provides shape and intensity features. We d e s c r ibe an approach for normalizing and mapping di erent modalities onto a common con dence interval. We also descri b e a n o vel method for integrating the scores of multiple classi ers. Veri cation experiments are reported for the individual modalities and for t he combined classier. The performance of the integrated system outperformed each subsystem and reduced the false acceptance rate of the acoustic subsystem from 2.3% to 0.5%.

This paper describes a fusion of visual and thermal infrared (IR) images for robust face recognition. Two types of fusion methods are discussed: data fusion and decision fusion. Data fusion produces an illumination-invariant face image by adaptively integrating registered visual and thermal face images. Decision fusion combines matching scores of individual face recognition modules. In the data fusion process, eyeglasses, which block thermal energy, are detected from thermal images and replaced with an eye template. Three fusion-based face recognition techniques are implemented and tested: Data fusion of visual and thermal images (Df), Decision fusion with highest matching score (Fh), and Decision fusion with average matching score (Fa). A commercial face recognition software FaceIt ® is used as an individual recognition module. Comparison results show that fusion-based face recognition techniques outperformed individual visual and thermal face recognizers under illumination variations and facial expressions.

In this paper, we address the multimodal biometric decision fusion problem. By exploring into the user-specific approach for learning and threshold setting, four possible paradigms for learning and decision making are investigated. Since each user requires a decision hyperplane specific to him in order to achieve good verification accuracy, those tedious iterative training methods like the neural network approach would not be suitable. We propose to use a model that requires only a single training step for this application. The four global and local learning and decision paradigms are then explored to observe their decision capability. Besides the proposal of a relevant receiver operating characteristic performance for the local decision, extensive experiments were conducted to observe the verification performance for fusion of two and three biometrics.

The contribution of this paper is to compare paradigms coming from the classes of parametric, and non-parametric techniques to solve the decision fusion problem encountered in the design of a multi-modal biometrical identity veri®cation system. The multimodal identity veri®cation system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, called score, stating how well the claimed identity is veri®ed. A decision fusion module receiving as input the d scores has to take a binary decision: accept or reject the claimed identity. We have solved this fusion problem using parametric and non-parametric classi®ers. The performances of all these fusion modules have been evaluated and compared with other approaches on a multi-modal database, containing both vocal and visual biometric modalities. Ó

In this paper we provide a study of channel-aware
decision fusion (DF) over a “virtual” multiple-input multiple-output (MIMO) channel in the large-array regime at the DF
center (DFC). The considered scenario takes into account channel estimation and inhomogeneous large-scale fading between the sensors and the DFC. The aim is the development of (widely) linear fusion rules, as opposed to the unsuitable optimum log-likelihood ratio (LLR). The proposed rules can effectively benefit from performance improvement via a large-array, differently from existing sub-optimal alternatives. Performance evaluation, along with theoretical achievable performance and complexity analysis, is presented. Simulation results are provided to confirm the findings. Analogies and differences with uplink communication in a multi-user (massive) MIMO scenario are underlined.

We study channel-aware binary-decision fusion over a shared Rayleigh flat-fading channel with multiple antennas at the Decision Fusion Center (DFC). We present the optimal rule and derive sub-optimal fusion rules, as alternatives with improved numerical stability, reduced complexity and lower system knowledge required. The set of rules is derived following both "Decode-and-Fuse" and "Decode-then-Fuse" approaches. Simulation results for performances are presented both under Neyman-Pearson and Bayesian frameworks. The effect of multiple antennas at the DFC for the presented rules is analyzed, showing corresponding benefits and limitations. Also, the effect on performances as a function of the number of sensors is studied under a total power constraint.

Two research strands, each identifying an area of markedly increasing importance in the current development of pattern analysis technology, underlie the review covered by this paper, and are drawn together to offer both a task-oriented and a fundamentally generic perspective on the discipline of pattern recognition. The first of these is the concept of decision fusion for high-performance pattern recognition, where (often very diverse) classification technologies, each providing complementary sources of information about class membership, can be integrated to provide more accurate, robust and reliable classification decisions. The second is the rapid expansion in technology for the automated analysis of (especially) handwritten data for OCR applications including document and form processing, pen-based computing, forensic analysis, biometrics and security, and many other areas, especially those which seek to provide online or offline processing of data which is available in a human-oriented medium. Classifier combination/multiple expert processing has a long history, but the sheer volume and diversity of possible strategies now available suggest that it is timely to consider a structured review of the field. Handwritten character processing provides an ideal context for such a review, both allowing engagement with a problem area which lends itself ideally to the performance enhancements offered by multi-classifier configurations, but also allowing a clearer focus to what otherwise, because of the unlimited application horizons, would be a task of unmanageable proportions. Hence, this paper explicitly reviews the field of multiple classifier decision combination strategies for character recognition, from some of its early roots to the present day. In order to give structure and a sense of direction to the review, a new taxonomy for categorising approaches is defined and explored, and this both imposes a discipline on the presentation of the material available and helps to clarify the mechanisms by which multi-classifier configurations deliver performance enhancements. The review incorporates a discussion both of processing structures themselves and a range of important related topics which are essential to maximise an understanding of the potential of such structures. Most importantly, the paper illustrates explicitly how the principles underlying the application of multi-classifier approaches to character recognition can easily generalise to a wide variety of different task domains.

Identification of humans from arbitrary view points is an important requirement for different tasks including perceptual interfaces for intelligent environments, covert security and access control etc. For optimal performance, the system must use as many cues as possible and combine them in meaningful ways. In this paper we present fusion of face and gait cues for the single camera case. We employ a view invariant gait recognition algorithm for gait recognition. A sequential importance sampling based algorithm is used for probabilistic face recognition from video. We employ decision fusion to combine the results of our gait recognition algorithm and the face recognition algorithm. We consider two fusion scenarios: hierarchical and holistic. The first involves using the gait recognition algorithm as a filter to pass on a smaller set of candidates to the face recognition algorithm. The second involves combining the similarity scores obtained individually from the face and gait recognition algorithms Simple rules like the SUM, MIN and PRODUCT are used for combinining the scores. The results of the fusion are demonstrated on the NIST database which has outdoor gait and face data of 30 subjects.

For a wireless sensor network (WSN) with a large number of sensors, a decision fusion rule using the total number of detections reported by local sensors for hypothesis testing, is proposed and studied. Based on a signal attenuation model where the received signal power decays as the distance from the target increases, the system level detection performance, namely probabilities of detection and false alarms, are derived and calculated. Without the knowledge of local sensors' performances and at low signal to noise ratio (SNR), this fusion rule can still achieve very good system level detection performance if the number of sensors is sufficiently large. The problem of designing an optimum local sensor level threshold is investigated. For various system parameters, the optimal thresholds are found numerically. Guidelines on selecting the optimal local threshold have been presented.

In this work we demonstrate an improvement in the state-of-theart large vocabulary continuous speech recognition (LVCSR) performance, under clean and noisy conditions, by the use of visual information, in addition to the traditional audio one. We take a decision fusion approach for the audio-visual information, where the single-modality (audio-and visual-only) HMM classifiers are combined to recognize audio-visual speech. More specifically, we tackle the problem of estimating the appropriate combination weights for each of the modalities. Two different techniques are described: The first uses an automatically extracted estimate of the audio stream reliability in order to modify the weights for each modality (both clean and noisy audio results are reported), while the second is a discriminative model combination approach where weights on pre-defined model classes are optimized to minimize WER (clean audio only results).

This paper provides an outline of a formalization of classes of information fusion systems in terms of category theory and formal languages. The formalization captures both the inputs/outputs of a fusion system and the fusion processing algorithms. The paper also introduces a notion of subclass, which is used to compare classes of fusion systems, whether they are different or one is a special case of another. Two examples of classes of fusion systems formalized in the paper are data fusion and decision fusion; decision fusion is shown to be a subclass of data fusion. A number of other classes of fusion systems are defined. The formalization is extended by adding the notion of measure of effectiveness, which is then used to prove that one of the classes (so called overlapping system) is at least as efficient as a single-source system. And finally it is shown how data association can be formalized in this framework. While at first the formalization could be used by information fusion scientists to formally define various types of fusion systems and then to prove theorems about properties of such systems, it is expected that it should lead to the development of tools that could be used by software engineers to formally derive designs of fusion systems.

We consider a decentralized multi-sensor estimation problem where L sensor nodes observe noisy versions of a correlated random source vector. The sensors amplify and forward their observations over a fading coherent multiple access channel (MAC) to a fusion center (FC). The FC is equipped with a large array of N antennas, and adopts a minimum mean square error (MMSE) approach for estimating the source. We optimize the amplification factor (or equivalently transmission power) at each sensor node in two different scenarios: a) with the objective of total power minimization subject to mean square error (MSE) of source estimation constraint, and b) with the objective of minimizing MSE subject to total power constraint. For this purpose, based on the well-known favorable propagation condition (when L ≪ N) achieved in massive multiple-input multiple-output (MIMO), we apply an asymptotic approximation on the MSE, and use convex optimization techniques to solve for the optimal sensor power allocation in a) and b). In a), we show that the total power consumption at the sensors decays as 1/N , replicating the power savings obtained in massive MIMO mobile communications literature. We also show several extensions of the aforementioned scenarios to the cases where sensor-to-FC fading channels are correlated, and channel coefficients are subject to estimation error. Through numerical studies, we also illustrate the superiority of the proposed optimal power allocation methods over uniform power allocation.

Recently, decision level fusion has shown great potential to increase classification accuracy beyond the level reached by individual classifiers. A considerable body of literature exists on identifying optimal ways to combine classifiers. However, the selection of the classifiers to be combined is equally, if not more, crucial if an improvement is to be made for certain classifier combination schemes. Agreement among classifiers can inhibit the gains obtained regardless of the method used to combine them. In this work, the level of agreement between different classifiers used in remote sensing is assessed based on statistical measures. A study is performed in which an image is classified by several methods with different degrees of agreement between them. The results are then combined using decision fusion schemes and the increase of accuracy is observed for each combination of the individual classifications.

Recognizing activities in a home environment is challenging due to the variety of activities that can be performed at home and the complexity of the environment. Multiple cameras are usually needed to cover the whole observation area. This adds camera fusion as another challenge to activity recognition. We propose a hierarchical approach that recognizes both coarse-level and fine-level activities, in which different image features and learning methods are used for different activities based on their characteristics. The paper focuses on discussing the second-level of activity recognition with spatio-temporal features. Specifically, three fusion approaches for multiview activity recognition with spatio-temporal features are presented, including two decision fusion methods and one feature fusion method. They are comparatively analyzed in terms of their tradeoffs on assumptions on system setup, model transferability and recognition rate. Experiments show that challenging activities wit...

We address the problem of optimizing the detection performance of sensor networks under communication constraints on the common access channel. Our work helps understanding tradeoffs between sensor network parameters like number of sensors, degree of quantization at each local sensor, and SNR. Traditionally, this problem is tackled using asymptotic assumptions on the number of sensors, an approach that leads to the abstraction of important details such as the structure of the fusion center. We adopt a non-asymptotic approach and optimize both, the sensing and the fusion sides with respect to the probability of detection error. We show that the optimal fusion rule has an interesting structure similar to the majority-voting rule. In addition, we study the convergence with respect to the number of sensors of the performance of the fusion rule. We show that convergence is SNR dependent and that, in low-SNR environments, asymptotics may require a large number of sensors.

Target classification fusion problem in a distributed, wireless sensor network is investigated. We propose a distance-based decision fusion scheme exploiting the relationship between sensor to target distance, signal to noise ratio and classification rate, which requires less communication while achieving higher region classification rate when compared to conventional majority-vote based fusion schemes. Several different methods are tested, and very encouraging simulation results using real world experimental data samples are also observed.

Intelligent Crowd Monitoring and Management Systems (ICMMSs) have become effective resources for strengthening safety and security along with enhancing early-warning capabilities to manage emergencies in crowded situations of smart cities and massive gatherings events. The main advantage of such systems is their ability to detect multiple features associated with the crowd gathering, as they enable multi-source sensors, multi-modal data, and powerful intelligent and analytical methods. Unlike traditional crowd monitoring systems, which make use of simplex forms of different data types, data and information associated with crowded scenarios can be collected, fused, processed and analyzed in large quantities for accurate global assessment and enhanced decision making processes in an ICMMS. Therefore, data fusion is introduced as an enabler to decrease data quantity, reduce data dimensions, and improve data quality. In this paper, we first survey the literature on data fusion application in crowd monitoring systems as we are developing a state-of-the-art ICMMS with data fusion as a major platform enabler. Next, we discuss some popular data fusion architectures and classifications from different perspectives. Based on this, we propose a multi-sensor, multi-modal, and dimensional ICMMS architecture based on data fusion. Then, we identify the data fusion processes in the ICMMS and classify them into sensor fusion, feature-based data fusion, and decision fusion. Relevant algorithms, applications and examples of three classes are elaborated. Finally, future data fusion research directions are discussed.

Consider the problem of signal detection via multiple distributed noisy sensors. We propose a linear decision fusion rule to combine the local statistics from individual sensors into a global statistic for binary hypothesis testing. The objective is to maximize the probability of detection subject to an upper limit on the probability of false alarm. We employ a divide-and-conquer strategy to divide the decision optimization problem into two subproblems, each of which is a nonconvex program with a quadratic constraint. Through a judicious reformulation and by employing a special matrix decomposition technique, we show that the two nonconvex subproblems can be solved by semidefinite programs in a globally optimal fashion. Hence, we can obtain the optimal linear fusion rule for the distributed detection problem. Compared with the likelihood-ratio test approach, the optimized linear fusion rule can achieve comparable detection performance with considerable design flexibility and reduced complexity.

Efficient decision fusion strategies, for deriving optimal decisions in multisensor target recognitionltracking environments, are postulated and analyzed in this study. This fusion paradigm, which is designed for fusing decisions derived from a suite of parallel sensors, is embedded within a recursive system structure to achieve significant enhancement in the reliability of the fused decisions.

This letter is focused on the classic problem of testing samples drawn from independent Bernoulli probability mass functions, when the success probability under the alternative hypothesis is not known. The goal is to provide a systematic taxonomy of the viable detectors (designed according to theoretically-founded criteria) which can be used for the specific instance of the problem. Both One-Sided (OS) and Two-Sided (TS) tests are considered, with reference to: (i) identical success
probability (a homogeneous scenario) or (ii) different success
probabilities (a non-homogeneous scenario) for the observed
samples. As a result of the study, a complete summary (in tabular form) of the relevant statistics for the problem is provided, along with a discussion on the existence of the Uniformly Most Powerful (UMP) test. Finally, when the Likelihood Ratio Test (LRT) is not UMP, existence of the UMP detector after reduction by invariance is investigated.

Received-energy test for non-coherent decision fusion over a Rayleigh fading multiple access channel (MAC) without diversity was recently shown to be optimum in the case of conditionally mutually independent and identically distributed (i.i.d.) sensor decisions under specific conditions [C. R. Berger, M. Guerriero, S. Zhou, and P. Willett, “PAC vs. MAC for Decentralized Detection Using Noncoherent Modulation,” IEEE Trans. Signal Process., vol. 57, no. 9, pp. 3562-2575, Sep. 2009], [F. Li, J. S. Evans, and S. Dey, “Decision Fusion Over Noncoherent Fading Multiaccess Channels,” IEEE Trans. Signal Process., vol. 59, no. 9, pp. 4367-4380, Sep. 2011]. Here, we provide a twofold generalization, allowing sensors to be non identical on one hand and introducing diversity on the other hand. Along with the derivation, we provide also a general tool to verify optimality of the received energy test in scenarios with correlated sensor decisions. Finally, we derive an analytical expression of the effect of the diversity on the large-system performances, under both individual and total power constraints.

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to L2-regularized Collaborative Representation Classifier (L2-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of L2-CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.

This paper describes a process for aggregating different information sources to estimate remaining equipment life. Specifically, the approach presents a rigorous chain of preprocessing, modeling and postprocessing steps that arrive at the desired prognostic result. The preprocessing steps deal with data reduction, filtering, and signature amplification. The prediction model applies Adaptive Neuro-Fuzzy Inference System (ANFIS) to the data. The post-processing steps include recursive trending which implicitly forces the prognostic trend to be confirmed before updated estimates are reported. Prognostic false positives and false negatives are introduced as innovative measures that help in assessing the performance of the approach. The method is illustrated using real-life data from industrial web paper breakage prediction.

We present a methodology of reliability estimation in the multimodal biometric verification scenario. Reliability estimation has shown to be an efficient and accurate way of predicting and correcting erroneous classification decisions in both unimodal (speech, face, online signature) and multimodal (speech and face) systems. While the initial research results indicate the high potential of the proposed methodology, the performance of the reliability estimation in a multimodal setting has not been sufficiently studied or evaluated. In this paper, we demonstrate the advantages of using the unimodal reliability information in order to perform an efficient biometric fusion of two modalities. We further show the presented method to be superior to state-of-the-art multimodal decision-level fusion schemes. The experimental evaluation presented in this paper is based on the popular benchmarking bimodal BANCA database.

In this paper, a video based algorithm for fire and flame detection is developed. In addition to ordinary motion and color clues, flame flicker is distinguished from motion of flame colored moving objects using Markov models. Irregular nature of flame boundaries is detected by performing temporal wavelet analysis using Hidden Markov Models as well. Color variations in fire is detected by computing the spatial wavelet transform of moving fire-colored regions. Boundary of flames are represented in wavelet domain and irregular nature of the boundaries of fire regions is also used as an indication of the flame flicker. Decisions from sub-algorithms are linearly combined using an adaptive active fusion method. The main detection algorithm is composed of four sub-algorithms (i) detection of fire colored moving objects, (ii) temporal, and (iii) spatial wavelet analysis for flicker detection and (iv) contour analysis of fire colored region boundaries. Each algorithm yields a continuous decision value as a real number in the range [-1, 1] at every image frame of a video sequence. Decision values from sub-algorithms are fused using an adaptive algorithm in which weights are updated using the least mean square (LMS) method in the training (learning) stage.