Optical flow and its extensions have been widely used in motion detection and computer vision. In... more Optical flow and its extensions have been widely used in motion detection and computer vision. In the study, principal component analysis (PCA) is applied to analyze optical flows for better motion detection performance. The joint optical flow and PCA approach can efficiently detect moving objects and suppress small turbulence. It is effective in both static and dynamic background. It is particularly useful for motion detection from outdoor videos with low quality and small moving objects. Experimental results demonstrate that this approach outperforms other existing methods by extracting the moving objects more completely with lower false alarms. Saving strategies are developed to reduce computational complexity of optical flow calculation and PCA. Graphic processing unit (GPU)-based parallel implementation is developed, which shows excellent speed up performance. ii DEDICATION I would like to dedicate this thesis to my beloved parents, Mr. Jian Liu and Mrs. Jianjun Chen, for their endless love. iii ACKNOWLEDGEMENTS I would like to express sincere thanks to my technical advisor, Dr. Jenny Q. Du, for her invaluable guidance and support for my research. I would also like to thank Dr. James Fowler and Dr. Pan Li for serving on my thesis committee and offering their advice to improve my work. I am grateful to Mr. He Yang and all my labmates for their assistance and cooperation during my school life at Mississippi State University.
This paper presents a real-time propagating and visualizing the uncertainty of multiple orbit sat... more This paper presents a real-time propagating and visualizing the uncertainty of multiple orbit satellites within the framework of graphics computing unit and multi-threading processing. The paper presents a system to predict the future position and velocity of orbiting objects based on a Monte Carlo method with multi-threading (Central Processing Unit (CPU)) and multi-stream (Graphics Processing Unit (GPU)) processing for the application of multiple satellite orbits estimation, collision probability calculation and visualization. The introduced SATellite Uncertinty Processing with GpU and Multi-threading (SAT-UPGUM) approach is general purpose in the sense that it can extended to other types of Space Situation Awareness (SSA) applications such as Stochastic Collocation, Pursui-Evasion, and Patterns of Life. The GPU based computing leads to a real-time outcome of orbit satellites uncertainty propagation and visualization compared to the situations when the whole work flow is applied in CPU only. The obtained propagation results for the multiple satellite orbits indicate that our GPU and multi-threading based approach provides dramatically improved frame rate under realistic conditions.
Sparse representation-based classification (SRC) has gained great interest recently. A pixel to b... more Sparse representation-based classification (SRC) has gained great interest recently. A pixel to be classified is sparsely approximately by labeled samples, and it is assigned to the class whose labeled samples provide the smallest representation error. In this paper, we extend SRC by exploiting the benefits of using a smoothing filter based on sparse gradient minimization. The smoothing filter is expected to provide less intra class variability and more spatial regularity, which eliminating the inherent variations within a small neighborhood. Classification performance on two real hyperspectral datasets demonstrates that our proposed method has improved classification accuracy and the resulting accuracies are persistently higher at all small training sample size situations compared to some traditional classifiers.
Detecting a target with low-occurrence-probability from unknown background in a hyperspectral ima... more Detecting a target with low-occurrence-probability from unknown background in a hyperspectral image, namely anomaly detection, is of practical significance. Reed-Xiaoli (RX) algorithm is considered as a classic anomaly detector, which calculates the Mahalanobis distance between local background and the pixel under test. Local RX, as an adaptive RX detector, employs a dual-window strategy to consider pixels within the frame between inner and outer windows as local background. However, the detector is sensitive if such a local region contains anomalous pixels (i.e., outliers). In this paper, a locality-constrained anomaly detector is proposed to remove outliers in the local background region before employing the RX algorithm. Specifically, a local linear representation is designed to exploit the internal relationship between linearly correlated pixels in the local background region and the pixel under test and its neighbors. Experimental results demonstrate that the proposed detector improves the original local RX algorithm.
Spatial texture features have been demonstrated to be very useful for the recently-proposed repre... more Spatial texture features have been demonstrated to be very useful for the recently-proposed representation-based classifiers, such as sparse representation-based classifier (SRC) and nearest regularized subspace (NRS). In this work, a weighted residual-fusion-based strategy with multiple features is proposed for these classifiers. Multiple features include local binary patterns (LBP), Gabor features, and the original spectral signatures. In the proposed classification framework, representation residuals for a testing pixel from using each type of features are weighted to generate the final representation residual, and then the label of the testing pixel is determined according to the class yielding the minimum final residual. The motivation of this work is that different features represent pixels from different perspectives and their fusion in the residual domain can enhance the discriminative ability. Experimental results of several real hyperspectral image datasets demonstrate that the proposed residual-based fusion outperforms the original NRS, SRC, support vector machine (SVM) with LBP, and SVM with Gabor features, even in small-sample-size (SSS) situations.
Computer Vision and Pattern Recognition Workshops(CVPRW), Jun 2015
This paper presents a Cloud-based architecture for detecting and tracking multiple moving targets... more This paper presents a Cloud-based architecture for detecting and tracking multiple moving targets from airborne videos combined with the audio assistance, which is called Cloud-based Audio-Video (CAV) fusion. The CAV system innovation is a method for user-based voice-to-text color feature descriptor track matching with an automated hue feature extraction from image pixels. The introduced CAV approach is general purpose for detecting and tracking different valuable targets' movement for suspicious behavior recognition through multi-intelligence data fusion. Using Cloud computing leads to real-time performance as compared a single machine workflow. The obtained multiple moving target tracking results from airborne videos demonstrate that the CAV approach provides improved frame rate, enhanced detection, and real-time tracking and classification performance under realistic conditions.
The use of hand gesture recognition has been steadily growing in various human-computer interacti... more The use of hand gesture recognition has been steadily growing in various human-computer interaction applications. Under realistic operating conditions, it has been shown that hand gesture recognition systems exhibit recognition rate limitations when using a single sensor. Two dual-sensor approaches have thus been developed in this dissertation in order to improve the performance of hand gesture recognition under realistic operating conditions. The first approach involves the use of image pairs from a stereo camera setup by merging the image information from the left and right camera, while the second approach involves the use of a Kinect depth camera and an inertial sensor by fusing differing modality data within the framework of a hidden Markov model. The emphasis of this dissertation has been on system building and practical deployment. More specifically, the major contributions of the dissertation are: (a) improvement of hand gestures recognition rates when using a pair of images from a stereo camera compared to when using a single image by fusing the information from the left and right images in a complementary manner, and (b) improvement of hand gestures recognition rates when using a dual-modality sensor setup consisting of a Kinect depth camera and an inertial body sensor compared to the situations when each sensor is used individually on its own. Experimental results obtained indicate that the developed approaches generate higher recognition rates in different backgrounds and lighting conditions compared to the situations when an individual sensor is used. Both approaches are designed such that the entire recognition system runs in real-time on PC platform.
Circuits and Systems Conference (DCAS), 2014 IEEE Dallas, Oct 12, 2014
This paper presents a multi-Hidden Markov Model (HMM) classification approach for hand gesture re... more This paper presents a multi-Hidden Markov Model (HMM) classification approach for hand gesture recognition by utilizing two differing modality and low-cost sensors. The sensors consist of a Kinect depth camera and a wearable inertial sensor. It is shown that the multi-HMM classification based on nine signals that are simultaneously captured by these two sensors leads to a more robust recognition compared to the situation when only a single HMM classification is used to generate the likelihood probabilities of hand gestures. This approach is applied to the hand gestures of the $1Unistroke Recognizer application and the results obtained indicate a 7% improvement in the overall classification rate over a single HMM classification under realistic conditions.
Journal of Applied Remote Sensing (JARS), Jan 1, 2014
A wavelet-based nearest-regularized-subspace classifier is proposed for noise-robust hyperspectra... more A wavelet-based nearest-regularized-subspace classifier is proposed for noise-robust hyperspectral image (HSI) classification. The nearest-regularized subspace, coupling the nearest- subspace classification with a distance-weighted Tikhonov regularization, was designed to only consider the original spectral bands. Recent research found that the multiscale wavelet features [e.g., extracted by redundant discrete wavelet transformation (RDWT)] of each hyperspectral pixel are potentially very useful and less sensitive to noise. An integration of wavelet-based features and the nearest-regularized-subspace classifier to improve the classification performance in noisy environments is proposed. Specifically, wealthy noise-robust features provided by RDWT based on hyperspectral spectrum are employed in a decision-fusion system or as preprocessing for the nearest-regularized-subspace (NRS) classifier. Improved performance of the proposed method over the conventional approaches, such as support vector machine, is shown by testing several HSIs. For example, the NRS classifier performed with an accuracy of 65.38% for the AVIRIS Indian Pines data with 75 training samples per class under noisy conditions (signal-to-noise ratio ¼ 36.87 dB), while the wavelet-based classifier can obtain an accuracy of 71.60%, resulting in an improvement of approximately 6%.
This paper presents a comparison of two real-time hand gesture recognition systems. One system ut... more This paper presents a comparison of two real-time hand gesture recognition systems. One system utilizes a binocular stereo camera set-up while the other system utilizes a combination of a depth camera and an inertial sensor. The latter system is a dual-modality system as it utilizes two different types of sensors. These systems have been previously developed in the Signal and Image Processing Laboratory at the University of Texas at Dallas and the details of the algorithms deployed in these systems are reported in previous papers. In this paper, a comparison is carried out between these two real-time systems in order to examine which system performs better for the same set of hand gestures under realistic conditions.
Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, Aug 26, 2014
This paper presents a home-based Senior Fitness Test (SFT) measurement system by using an inertia... more This paper presents a home-based Senior Fitness Test (SFT) measurement system by using an inertial sensor and a depth camera in a collaborative way. The depth camera is used to monitor the correct pose of a subject for a fitness test and any deviation from the correct pose while the inertial sensor is used to measure the number of a fitness test action performed by the subject within the time duration specified by the fitness protocol. The results indicate that this collaborative approach leads to high success rates in providing the SFT measurements under realistic conditions.
Journal of Real-Time Image Processing (JRTIP), Feb 11, 2013
This paper presents a real-time and robust approach to recognize two types of gestures consisting... more This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic
lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.
Extreme learning machine (ELM) is a single-layer feedforward neural network based classifier that... more Extreme learning machine (ELM) is a single-layer feedforward neural network based classifier that has attracted significant attention in computer vision and pattern recognition due to its fast learning speed and strong generalization. In this paper, we propose to integrate spectral-spatial information for hyperspectral image classification and exploit the benefits of using spatial features for the kernel based ELM (KELM) classifier. Specifically, Gabor filtering and multihypothesis (MH) prediction preprocessing are two approaches employed for spatial feature extraction. Gabor features have currently been successfully applied for hyperspectral image analysis due to the ability to represent useful spatial information. MH prediction preprocessing makes use of the spatial piecewise-continuous nature of hyperspectral imagery to integrate spectral and spatial information. The proposed Gabor-filtering-based KELM classifier and MH-prediction-based KELM classifier have been validated on two real hyperspectral datasets. Classification results demonstrate that the proposed methods outperform the conventional pixel-wise classifiers as well as Gabor-filtering-based support vector machine (SVM) and MH-prediction-based SVM in challenging small training sample size conditions.
Journal of Real-Time Image Processing (JRTIP), Aug 2013
This paper presents a human action recognition method by using depth motion maps. Each depth fram... more This paper presents a human action recognition method by using depth motion maps. Each depth frame in a depth video sequence is projected onto three orthogonal Cartesian planes. Under each projection view, the absolute difference between two consecutive projected maps is accumulated through an entire depth video sequence forming a depth motion map. An L2 regularized collaborative representation classifier with a distance-weighted Tikhonov matrix is then employed for action recognition. The developed method is shown to be computationally efficient allowing it to run in real-time. The recognition results applied to the Microsoft Research Action3D dataset indicate superior performance of our method over the existing methods
A novel approach using volumetric texture and reduced-spectral features is presented for hyperspe... more A novel approach using volumetric texture and reduced-spectral features is presented for hyperspectral image classification. Using this approach, the volumetric textural features were extracted by volumetric gray-level co-occurrence matrices (VGLCM). The spectral features were extracted by minimum estimated abundance covariance (MEAC) and linear prediction (LP)-based band selection, and a semi-supervised k-means (SKM) clustering method with deleting the worst cluster (SKMd) bandclustering algorithms. Moreover, four feature combination schemes were designed for hyperspectral image classification by using spectral and textural features. It has been proven that the proposed method using VGLCM outperforms the gray-level co-occurrence matrices (GLCM) method, and the experimental results indicate that the combination of spectral information with volumetric textural features leads to an improved classification performance in hyperspectral imagery.
Journal of Applied Remote Sensing (JARS), Jan 1, 2014
This paper presents a new dynamic classifier selection approach for hyperspectral image classific... more This paper presents a new dynamic classifier selection approach for hyperspectral image classification, in which both spatial and spectral information are used to determine a pixel’s label once the remaining classified pixels’ neighborhood meets the threshold. For volumetric texture feature extraction, a volumetric gray level co-occurrence matrix is used; for spectral feature extraction, a minimum estimated abundance covariance-based band selection is used. Two hyperspectral remote sensing datasets, HYDICEWashington DC Mall and AVIRIS Indian Pines, are employed to evaluate the performance of the developed method. The classification accuracies of the two datasets are improved by 1.13% and 4.47%, respectively, compared with the traditional algorithms using spectral information. The experimental results demonstrate that the integration of spectral information with volumetric textural features can improve the classification performance for hyperspectral images.
Band selection is a commonly used approach for dimensionality reduction of hyperspectral imagery.... more Band selection is a commonly used approach for dimensionality reduction of hyperspectral imagery. Affinity propagation (AP), as the newly presented clustering algorithm, is addressed in many fields, and it also can be used for hyperspectral band selection. However this algorithm cannot get the fixed number of “exemplar” during message-passing procedure. This paper explores AP and proposes an adaptive AP (AAP) algorithm for semi-supervised hyperspectral band selection, and investigates distance metrics for improving band selection performance. In detail, the “exemplar” numbers determinant algorithm and bisection method is addressed to improve AP procedure, and the relations between selected “exemplar” numbers and “preferences” are established. Experiments are conducted to evaluate the proposed AAP-based band selection algorithm; the results demonstrate that the proposed method can outperform other popular methods with lower computational cost.
Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, Mar 14, 2010
Optical flow and its extensions have been widely used in motion detection and computer vision. In... more Optical flow and its extensions have been widely used in motion detection and computer vision. In this paper, we apply principle component analysis (PCA) to analyze optical flows for better motion detection performance. The joint optical flow and PCA approach can efficiently detect moving objects and suppress small turbulence. It is effective in both static and dynamic background. It is particularly useful for motion detection from outdoor videos with low quality and small moving objects. Preliminary results demonstrate that this approach outperforms other existing methods by extracting the moving objects more completely with lower false alarms.
This paper presents the first attempt at fusing data from inertial and vision depth sensors withi... more This paper presents the first attempt at fusing data from inertial and vision depth sensors within the framework of a Hidden Markov Model (HMM) for the application of hand gesture recognition. The data fusion approach introduced in this paper is general purpose in the sense that it can be used for recognition of various body movements. It is shown that the fusion of data from the vision depth and inertial sensors act in a complementary manner leading to a more robust recognition outcome compared to the situations when each sensor is used individually on its own. The obtained recognition rates for the single hand gestures in the Microsoft MSR dataset indicate that our fusion approach provides improved recognition in real-time and under realistic conditions.
Optical flow and its extensions have been widely used in motion detection and computer vision. In... more Optical flow and its extensions have been widely used in motion detection and computer vision. In the study, principal component analysis (PCA) is applied to analyze optical flows for better motion detection performance. The joint optical flow and PCA approach can efficiently detect moving objects and suppress small turbulence. It is effective in both static and dynamic background. It is particularly useful for motion detection from outdoor videos with low quality and small moving objects. Experimental results demonstrate that this approach outperforms other existing methods by extracting the moving objects more completely with lower false alarms. Saving strategies are developed to reduce computational complexity of optical flow calculation and PCA. Graphic processing unit (GPU)-based parallel implementation is developed, which shows excellent speed up performance. ii DEDICATION I would like to dedicate this thesis to my beloved parents, Mr. Jian Liu and Mrs. Jianjun Chen, for their endless love. iii ACKNOWLEDGEMENTS I would like to express sincere thanks to my technical advisor, Dr. Jenny Q. Du, for her invaluable guidance and support for my research. I would also like to thank Dr. James Fowler and Dr. Pan Li for serving on my thesis committee and offering their advice to improve my work. I am grateful to Mr. He Yang and all my labmates for their assistance and cooperation during my school life at Mississippi State University.
This paper presents a real-time propagating and visualizing the uncertainty of multiple orbit sat... more This paper presents a real-time propagating and visualizing the uncertainty of multiple orbit satellites within the framework of graphics computing unit and multi-threading processing. The paper presents a system to predict the future position and velocity of orbiting objects based on a Monte Carlo method with multi-threading (Central Processing Unit (CPU)) and multi-stream (Graphics Processing Unit (GPU)) processing for the application of multiple satellite orbits estimation, collision probability calculation and visualization. The introduced SATellite Uncertinty Processing with GpU and Multi-threading (SAT-UPGUM) approach is general purpose in the sense that it can extended to other types of Space Situation Awareness (SSA) applications such as Stochastic Collocation, Pursui-Evasion, and Patterns of Life. The GPU based computing leads to a real-time outcome of orbit satellites uncertainty propagation and visualization compared to the situations when the whole work flow is applied in CPU only. The obtained propagation results for the multiple satellite orbits indicate that our GPU and multi-threading based approach provides dramatically improved frame rate under realistic conditions.
Sparse representation-based classification (SRC) has gained great interest recently. A pixel to b... more Sparse representation-based classification (SRC) has gained great interest recently. A pixel to be classified is sparsely approximately by labeled samples, and it is assigned to the class whose labeled samples provide the smallest representation error. In this paper, we extend SRC by exploiting the benefits of using a smoothing filter based on sparse gradient minimization. The smoothing filter is expected to provide less intra class variability and more spatial regularity, which eliminating the inherent variations within a small neighborhood. Classification performance on two real hyperspectral datasets demonstrates that our proposed method has improved classification accuracy and the resulting accuracies are persistently higher at all small training sample size situations compared to some traditional classifiers.
Detecting a target with low-occurrence-probability from unknown background in a hyperspectral ima... more Detecting a target with low-occurrence-probability from unknown background in a hyperspectral image, namely anomaly detection, is of practical significance. Reed-Xiaoli (RX) algorithm is considered as a classic anomaly detector, which calculates the Mahalanobis distance between local background and the pixel under test. Local RX, as an adaptive RX detector, employs a dual-window strategy to consider pixels within the frame between inner and outer windows as local background. However, the detector is sensitive if such a local region contains anomalous pixels (i.e., outliers). In this paper, a locality-constrained anomaly detector is proposed to remove outliers in the local background region before employing the RX algorithm. Specifically, a local linear representation is designed to exploit the internal relationship between linearly correlated pixels in the local background region and the pixel under test and its neighbors. Experimental results demonstrate that the proposed detector improves the original local RX algorithm.
Spatial texture features have been demonstrated to be very useful for the recently-proposed repre... more Spatial texture features have been demonstrated to be very useful for the recently-proposed representation-based classifiers, such as sparse representation-based classifier (SRC) and nearest regularized subspace (NRS). In this work, a weighted residual-fusion-based strategy with multiple features is proposed for these classifiers. Multiple features include local binary patterns (LBP), Gabor features, and the original spectral signatures. In the proposed classification framework, representation residuals for a testing pixel from using each type of features are weighted to generate the final representation residual, and then the label of the testing pixel is determined according to the class yielding the minimum final residual. The motivation of this work is that different features represent pixels from different perspectives and their fusion in the residual domain can enhance the discriminative ability. Experimental results of several real hyperspectral image datasets demonstrate that the proposed residual-based fusion outperforms the original NRS, SRC, support vector machine (SVM) with LBP, and SVM with Gabor features, even in small-sample-size (SSS) situations.
Computer Vision and Pattern Recognition Workshops(CVPRW), Jun 2015
This paper presents a Cloud-based architecture for detecting and tracking multiple moving targets... more This paper presents a Cloud-based architecture for detecting and tracking multiple moving targets from airborne videos combined with the audio assistance, which is called Cloud-based Audio-Video (CAV) fusion. The CAV system innovation is a method for user-based voice-to-text color feature descriptor track matching with an automated hue feature extraction from image pixels. The introduced CAV approach is general purpose for detecting and tracking different valuable targets' movement for suspicious behavior recognition through multi-intelligence data fusion. Using Cloud computing leads to real-time performance as compared a single machine workflow. The obtained multiple moving target tracking results from airborne videos demonstrate that the CAV approach provides improved frame rate, enhanced detection, and real-time tracking and classification performance under realistic conditions.
The use of hand gesture recognition has been steadily growing in various human-computer interacti... more The use of hand gesture recognition has been steadily growing in various human-computer interaction applications. Under realistic operating conditions, it has been shown that hand gesture recognition systems exhibit recognition rate limitations when using a single sensor. Two dual-sensor approaches have thus been developed in this dissertation in order to improve the performance of hand gesture recognition under realistic operating conditions. The first approach involves the use of image pairs from a stereo camera setup by merging the image information from the left and right camera, while the second approach involves the use of a Kinect depth camera and an inertial sensor by fusing differing modality data within the framework of a hidden Markov model. The emphasis of this dissertation has been on system building and practical deployment. More specifically, the major contributions of the dissertation are: (a) improvement of hand gestures recognition rates when using a pair of images from a stereo camera compared to when using a single image by fusing the information from the left and right images in a complementary manner, and (b) improvement of hand gestures recognition rates when using a dual-modality sensor setup consisting of a Kinect depth camera and an inertial body sensor compared to the situations when each sensor is used individually on its own. Experimental results obtained indicate that the developed approaches generate higher recognition rates in different backgrounds and lighting conditions compared to the situations when an individual sensor is used. Both approaches are designed such that the entire recognition system runs in real-time on PC platform.
Circuits and Systems Conference (DCAS), 2014 IEEE Dallas, Oct 12, 2014
This paper presents a multi-Hidden Markov Model (HMM) classification approach for hand gesture re... more This paper presents a multi-Hidden Markov Model (HMM) classification approach for hand gesture recognition by utilizing two differing modality and low-cost sensors. The sensors consist of a Kinect depth camera and a wearable inertial sensor. It is shown that the multi-HMM classification based on nine signals that are simultaneously captured by these two sensors leads to a more robust recognition compared to the situation when only a single HMM classification is used to generate the likelihood probabilities of hand gestures. This approach is applied to the hand gestures of the $1Unistroke Recognizer application and the results obtained indicate a 7% improvement in the overall classification rate over a single HMM classification under realistic conditions.
Journal of Applied Remote Sensing (JARS), Jan 1, 2014
A wavelet-based nearest-regularized-subspace classifier is proposed for noise-robust hyperspectra... more A wavelet-based nearest-regularized-subspace classifier is proposed for noise-robust hyperspectral image (HSI) classification. The nearest-regularized subspace, coupling the nearest- subspace classification with a distance-weighted Tikhonov regularization, was designed to only consider the original spectral bands. Recent research found that the multiscale wavelet features [e.g., extracted by redundant discrete wavelet transformation (RDWT)] of each hyperspectral pixel are potentially very useful and less sensitive to noise. An integration of wavelet-based features and the nearest-regularized-subspace classifier to improve the classification performance in noisy environments is proposed. Specifically, wealthy noise-robust features provided by RDWT based on hyperspectral spectrum are employed in a decision-fusion system or as preprocessing for the nearest-regularized-subspace (NRS) classifier. Improved performance of the proposed method over the conventional approaches, such as support vector machine, is shown by testing several HSIs. For example, the NRS classifier performed with an accuracy of 65.38% for the AVIRIS Indian Pines data with 75 training samples per class under noisy conditions (signal-to-noise ratio ¼ 36.87 dB), while the wavelet-based classifier can obtain an accuracy of 71.60%, resulting in an improvement of approximately 6%.
This paper presents a comparison of two real-time hand gesture recognition systems. One system ut... more This paper presents a comparison of two real-time hand gesture recognition systems. One system utilizes a binocular stereo camera set-up while the other system utilizes a combination of a depth camera and an inertial sensor. The latter system is a dual-modality system as it utilizes two different types of sensors. These systems have been previously developed in the Signal and Image Processing Laboratory at the University of Texas at Dallas and the details of the algorithms deployed in these systems are reported in previous papers. In this paper, a comparison is carried out between these two real-time systems in order to examine which system performs better for the same set of hand gestures under realistic conditions.
Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, Aug 26, 2014
This paper presents a home-based Senior Fitness Test (SFT) measurement system by using an inertia... more This paper presents a home-based Senior Fitness Test (SFT) measurement system by using an inertial sensor and a depth camera in a collaborative way. The depth camera is used to monitor the correct pose of a subject for a fitness test and any deviation from the correct pose while the inertial sensor is used to measure the number of a fitness test action performed by the subject within the time duration specified by the fitness protocol. The results indicate that this collaborative approach leads to high success rates in providing the SFT measurements under realistic conditions.
Journal of Real-Time Image Processing (JRTIP), Feb 11, 2013
This paper presents a real-time and robust approach to recognize two types of gestures consisting... more This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic
lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.
Extreme learning machine (ELM) is a single-layer feedforward neural network based classifier that... more Extreme learning machine (ELM) is a single-layer feedforward neural network based classifier that has attracted significant attention in computer vision and pattern recognition due to its fast learning speed and strong generalization. In this paper, we propose to integrate spectral-spatial information for hyperspectral image classification and exploit the benefits of using spatial features for the kernel based ELM (KELM) classifier. Specifically, Gabor filtering and multihypothesis (MH) prediction preprocessing are two approaches employed for spatial feature extraction. Gabor features have currently been successfully applied for hyperspectral image analysis due to the ability to represent useful spatial information. MH prediction preprocessing makes use of the spatial piecewise-continuous nature of hyperspectral imagery to integrate spectral and spatial information. The proposed Gabor-filtering-based KELM classifier and MH-prediction-based KELM classifier have been validated on two real hyperspectral datasets. Classification results demonstrate that the proposed methods outperform the conventional pixel-wise classifiers as well as Gabor-filtering-based support vector machine (SVM) and MH-prediction-based SVM in challenging small training sample size conditions.
Journal of Real-Time Image Processing (JRTIP), Aug 2013
This paper presents a human action recognition method by using depth motion maps. Each depth fram... more This paper presents a human action recognition method by using depth motion maps. Each depth frame in a depth video sequence is projected onto three orthogonal Cartesian planes. Under each projection view, the absolute difference between two consecutive projected maps is accumulated through an entire depth video sequence forming a depth motion map. An L2 regularized collaborative representation classifier with a distance-weighted Tikhonov matrix is then employed for action recognition. The developed method is shown to be computationally efficient allowing it to run in real-time. The recognition results applied to the Microsoft Research Action3D dataset indicate superior performance of our method over the existing methods
A novel approach using volumetric texture and reduced-spectral features is presented for hyperspe... more A novel approach using volumetric texture and reduced-spectral features is presented for hyperspectral image classification. Using this approach, the volumetric textural features were extracted by volumetric gray-level co-occurrence matrices (VGLCM). The spectral features were extracted by minimum estimated abundance covariance (MEAC) and linear prediction (LP)-based band selection, and a semi-supervised k-means (SKM) clustering method with deleting the worst cluster (SKMd) bandclustering algorithms. Moreover, four feature combination schemes were designed for hyperspectral image classification by using spectral and textural features. It has been proven that the proposed method using VGLCM outperforms the gray-level co-occurrence matrices (GLCM) method, and the experimental results indicate that the combination of spectral information with volumetric textural features leads to an improved classification performance in hyperspectral imagery.
Journal of Applied Remote Sensing (JARS), Jan 1, 2014
This paper presents a new dynamic classifier selection approach for hyperspectral image classific... more This paper presents a new dynamic classifier selection approach for hyperspectral image classification, in which both spatial and spectral information are used to determine a pixel’s label once the remaining classified pixels’ neighborhood meets the threshold. For volumetric texture feature extraction, a volumetric gray level co-occurrence matrix is used; for spectral feature extraction, a minimum estimated abundance covariance-based band selection is used. Two hyperspectral remote sensing datasets, HYDICEWashington DC Mall and AVIRIS Indian Pines, are employed to evaluate the performance of the developed method. The classification accuracies of the two datasets are improved by 1.13% and 4.47%, respectively, compared with the traditional algorithms using spectral information. The experimental results demonstrate that the integration of spectral information with volumetric textural features can improve the classification performance for hyperspectral images.
Band selection is a commonly used approach for dimensionality reduction of hyperspectral imagery.... more Band selection is a commonly used approach for dimensionality reduction of hyperspectral imagery. Affinity propagation (AP), as the newly presented clustering algorithm, is addressed in many fields, and it also can be used for hyperspectral band selection. However this algorithm cannot get the fixed number of “exemplar” during message-passing procedure. This paper explores AP and proposes an adaptive AP (AAP) algorithm for semi-supervised hyperspectral band selection, and investigates distance metrics for improving band selection performance. In detail, the “exemplar” numbers determinant algorithm and bisection method is addressed to improve AP procedure, and the relations between selected “exemplar” numbers and “preferences” are established. Experiments are conducted to evaluate the proposed AAP-based band selection algorithm; the results demonstrate that the proposed method can outperform other popular methods with lower computational cost.
Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, Mar 14, 2010
Optical flow and its extensions have been widely used in motion detection and computer vision. In... more Optical flow and its extensions have been widely used in motion detection and computer vision. In this paper, we apply principle component analysis (PCA) to analyze optical flows for better motion detection performance. The joint optical flow and PCA approach can efficiently detect moving objects and suppress small turbulence. It is effective in both static and dynamic background. It is particularly useful for motion detection from outdoor videos with low quality and small moving objects. Preliminary results demonstrate that this approach outperforms other existing methods by extracting the moving objects more completely with lower false alarms.
This paper presents the first attempt at fusing data from inertial and vision depth sensors withi... more This paper presents the first attempt at fusing data from inertial and vision depth sensors within the framework of a Hidden Markov Model (HMM) for the application of hand gesture recognition. The data fusion approach introduced in this paper is general purpose in the sense that it can be used for recognition of various body movements. It is shown that the fusion of data from the vision depth and inertial sensors act in a complementary manner leading to a more robust recognition outcome compared to the situations when each sensor is used individually on its own. The obtained recognition rates for the single hand gestures in the Microsoft MSR dataset indicate that our fusion approach provides improved recognition in real-time and under realistic conditions.
Uploads
Papers by Kui Liu
lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.
lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.