Two-dimensional FLD for face recognition

Pattern Recognition 38 (2005) 1121 – 1124 www.elsevier.com/locate/patcog Rapid and brief communication Two-dimensional FLD for face recognition夡 Huilin Xiong, M.N.S. Swamy∗ , M.O. Ahmad Department of Electrical and Computer Engineering, Concordia University, 1455 de Maisonneuve Blvd. West, Montreal, Quebec, Canada H3G 1M8 Received 24 November 2004; accepted 8 December 2004 Abstract This paper presents a new scheme of face image feature extraction, namely, the two-dimensional Fisher linear discriminant. Experiments on the ORL and the UMIST face databases show that the new scheme outperforms the PCA and the conventional PCA + FLD schemes, not only in its computational efficiency, but also in its performance for the task of face recognition. 䉷 2005 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Fisher criterion; Principal component analysis (PCA); Linear discriminant analysis (LDA) 1. Introduction Fisher linear discriminant (FLD), sometimes known as linear discriminant analysis (LDA), has been widely used in pattern recognition for feature extraction and dimension reduction. The objective of the FLD is to find the optimal projection so that the ratio of the determinants of the betweenclass and the within-class scatter matrices of the projected samples reaches its maximum. A difficulty in using the FLD method for face recognition is the very high-dimensional nature of the image vector. For an image of size 112 × 92, the commonly used image size in face recognition, the dimension of the vector space is 10304, and the size of the scatter matrices 10304 × 10304. Obviously, it is quite difficult to handle matrices of such large size. Moreover, the withinclass scatter matrix is always singular, making the direct implementation of the classical FLD algorithm impossible. 夡 This work was partially supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada. ∗ Corresponding author. Tel.: +1 514 848 2424x3091; fax: +1 514 848 2802. E-mail addresses: [email protected] (H. Xiong), [email protected] (M.N.S. Swamy), [email protected] (M.O. Ahmad). The traditional solution to this problem is to utilize the principal component analysis (PCA) as a pre-processing step aiming to reduce the dimensionality of the vector space. After all the image vectors are projected into the subspace consisting of the “principal components”, the FLD algorithm can perform well in the subspace. However, since the projection criterion of the PCA and that of the FLD are essentially different, the pre-processing procedure to reduce the dimensionality using the PCA could result in the loss of some important discriminatory information for the FLD algorithm that follows the PCA. Actually, Chen et al. [1] have shown that the null space of the within-class scatter matrix contains valuable discriminatory information. In view of this, the so-called direct LDA (DLDA) algorithms [1,2] have been proposed to avoid the possible loss of useful information. An alternative way to handle the above problem is to directly project the image matrix under a specific projection criterion, rather than using the stretched image vector. Yang et al. [3] have shown that a “two-dimensional” PCA (2DPCA) can be constructed in a straightforward manner based on the image matrix projection. The size of the scatter matrices for the 2DPCA scheme is either only m × m or n×n for an image of size m×n, instead of the size mn×mn in the classic PCA scheme. Therefore, the 2DPCA scheme 0031-3203/$30.00 䉷 2005 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2004.12.003 1122 H. Xiong et al. / Pattern Recognition 38 (2005) 1121 – 1124 is much faster in computation than the conventional PCA algorithm. The most interesting fact shown in Ref. [3] is that the features extracted by the 2DPCA scheme are more efficient than those extracted by the conventional PCA scheme for the task of face recognition and image reconstruction. In this paper, we present a two-dimensional FLD scheme, called 2DFLD, which employs an alternate to the commonly used measure of the Fisher discriminant scalar, and derive a matrix-projection-based FLD algorithm. Experimental results show that the 2DFLD feature extraction scheme outperforms the conventional PCA and PCA + FLD scheme not only in terms of the computational efficiency, but also in terms of the performance for face recognition. 2. Two-dimensional FLD for image feature extraction We project an m × n image matrix X onto an mdimensional vector space through the transformation y = X
, where
is an n-dimensional vector, and y the mdimensional projected vector. Our goal is to find the optimal projection direction
so that the projected vectors in the mdimensional space reaches its maximum class separability. 2.1. An alternative Fisher criterion The conventional Fisher criterion is not convenient for the theoretical analysis in the case of our matrix-based projection. Instead, in this paper, we adopt the alternative measure of class separability given by J= tr(Sb ) , tr(Sw ) tr(Sb
)
) tr(Sw
= Sw C 1  i i ni (X − X)T (X − X), N i=1 C 1   i i (Xj − X )T (Xj − X ) Gw = N i=1 j ∈Ci i in which X and X , respectively, represent the global and the ith class mean images. We call matrices Gb and Gw , image between-class scatter matrix and image within-class scatter matrix, respectively. Note that the size of the image scatter matrices is only m×m, much smaller than that of the scatter matrices whose sizes are mn × mn in the conventional FLD algorithm. Using the image scatter matrices, the two-dimensional Fisher criterion given by Eq. (1) can be expressed as
T Gb
. J (
) = T
Gw
(3) 2.2. Two-dimensional FLD feature extraction The goal of our 2DFLD scheme is to find the optimal projection direction
in order to maximize (3). Obviously, the optimal projection direction
opt is the eigenvector corresponding to the maximum eigenvalue of the eigenstructure: Gb
= Gw
. , (2) (4) It is not difficult to handle the above eigenproblem directly, since the size of the matrix Gb or Gw is only m × m. In practice, one optimal projective direction is not enough to extract sufficient discriminatory features. We usually need to project the image data onto a set of orthogonal directions, namely,
1 ,
2 , . . . ,
k , which maximize the criterion (3). These projection directions can be selected as the k eigenvectors corresponding to the first k largest eigenvalues of the eigenstructure (4). Suppose {
i }ki=1 are the optimal projective directions. Given an image X, all the projections of the image matrix in the k directions make up a mk-dimensional vector y, which is our 2DFLD feature vector T T T y T = (y1T , y2T , . . . , ykT ) = (
T 1 ,
2 , . . . ,
k )X . where Sb
= Gb = (1) where “tr” denotes the trace of a matrix, and Sb and Sw , respectively, denote the between-class and within-class matrices. The measure J in Eq. (1) is also a well-known Fisher scalar for measuring class separability, actually, it is called “criteria J4 ” in Ref. [4]. Suppose {Xi }N i=1 are the training images, which contain C
classes (subjects), and the ith class Ci has ni samples ( C i=1 ni =N ). The images, all m×n matrices, are projected into a m-dimensional vector space yi =Xi
. In the projection space, the measure of the class separability of the projected images is calculated by J (
) = in which y and y i , respectively, denote the global mean vector and the mean vector of the ith class in the projection space.
) = It is easy to verify that tr(Sb
) =
T Gb
and tr(Sw T
Gw
, where C 1  ni (y i − y)(y i − y)T , N i=1 C 1   N i=1 j ∈Ci (yj − y i )(yj − y i )T 3. Experimental results Two face image databases, namely, the ORL database and UMIST database, are used to compare the proposed 2DFLD approach with the following algorithms: the PCA, H. Xiong et al. / Pattern Recognition 38 (2005) 1121 – 1124 1123 Table 1 Comparison of the average error rates (%) of different approaches on the ORL database p 2 3 4 5 6 PCA PCA + FLD DLDA 2DPCA 2DFLD 18.42 21.41 19.98 15.81 13.41 11.57 14.14 11.80 10.68 8.34 8.13 10.58 8.23 7.10 6.54 5.68 9.02 5.45 4.73 4.60 4.78 7.03 4.03 3.78 3.70 Fig. 1. Ten sample images of a subject in the UMIST database. the PCA + FLD [5], the direct LDA (DLDA) [2], and the 2DPCA [3] schemes. While the ORL database is used to test the performance of the face recognition algorithms under the condition of minor variation of scaling and rotation, the UMIST database is used to examine the performance of the algorithms when the angle of rotation of the facial images is quite large. The ORL database (http://www.cam-orl.co.uk) contains 40 persons, each having 10 different images. The UMIST face image database, available at http://images.ee.umist.ac. uk/danny/database.html, contains 20 subjects, each covering a wide range of multiviewed face images, from profile to frontal views. We select 25 pictures from each subject to construct our UMIST input images. Note that there are six subjects in which the image number is less than 25, and a few “mirror” images generated by exchanging the right and left pixels of some existing frontal pictures are added to the subject in order to compliment the image number in the subject up to 25. Therefore, the total image number of the UMIST data set is 500. Fig. 1 shows ten sample images of one of the subject. In the experiments, we randomly select p images from each subject to construct the training data set, the remaining images being used as the test images. To ensure sufficient training, a value of at least 2 is used for p. Since k, the number of projection vectors, has a considerable impact on the results of the different algorithms, we choose the value that corresponds to the best classification result on the image set consisting of the first p images of each subject as its optimal value. In all of the experiments, the nearest neighbor algorithm under the Euclidean distance is employed to classify the test images. Each experiment is repeated 20 times. The average recognition error rates of the different algorithms on the test sets are, respectively, summarized in Table 1 for the ORL database, and Table 2 for the UMIST database. Table 3 shows the average CPU time (CPU: PIII 800, RAM: 128MB) taken for feature extraction on the ORL database under the Matlab 6.1 platform for the various algorithms. We see from these tables that the proposed 2DFLD approach is more efficient than the PCA and the PCA + FLD schemes, not only in terms of computation times, but also in terms of the error rate for face recognition. The proposed 2DFLD approach Table 2 Comparison of different approaches in terms of error rates (%) on the UMIST database p 2 3 4 5 6 7 PCA PCA + FLD DLDA 2DPCA 2DFLD 32.08 29.90 28.28 22.76 19.70 24.70 18.30 18.34 16.42 12.75 16.94 14.16 13.25 9.51 14.24 10.98 9.33 8.26 11.62 9.74 8.99 6.04 10.39 7.98 6.86 4.47 7.11 5.91 5.46 3.08 8 6.40 5.43 3.79 3.07 1.76 Table 3 Comparison of the average CPU time (s) for feature extraction on the ORL database p 2 3 4 5 6 PCA PCA + FLD DLDA 2DPCA 2DFLD 57.49 34.43 33.44 3.17 3.91 77.42 67.68 33.29 4.30 4.40 106.72 90.44 32.63 4.10 4.78 74.34 140.31 37.36 5.11 5.87 66.37 164.43 36.98 5.28 6.26 also performs better than the DLDA and 2DPCA schemes in terms of the error rate, although the 2DFLD scheme is comparable to the 2DPCA scheme in terms of the computational efficiency for the feature extraction. 4. Conclusion In this paper, we have presented a new scheme for image feature extraction, namely, the two dimensional FLD (2DFLD), which is based on a straightforward projection of the image matrix along with an alternative Fisher criterion. Since the size of the scatter matrices in the 2DFLD scheme is much smaller than those in the conventional FLD and PCA schemes, the computation of the feature extraction using the proposed 2DFLD scheme is much more efficient than that of the PCA and PCA + FLD schemes. More importantly, experimental results on the ORL and UMIST databases have shown that the 2DFLD features outperform the features extracted by the PCA, 2DPCA, PCA+FLD, and DLDA schemes in the performance of face recognition. 1124 H. Xiong et al. / Pattern Recognition 38 (2005) 1121 – 1124 References [1] L. Chen, H. Liao, M. Ko, J. Lin, G. Yu, A new LDA-based face recognition which can solve the small sample size problem, Pattern Recogn. 33 (10) (2000) 1713–1726. [2] H. Yu, J. Yang, A direct LDA algorithm for high-dimensional data–with application to face recognition, Pattern Recogn. 34 (2001) 2067–2070. [3] J. Yang, D. Zhang, A.F. Frangi, J.-Y. Yang, Two-dimensional pca: a new approach to appearance-based face representation and recognition, IEEE Trans. Pattern Anal. Mach. Intell. 26 (1) (2004) 131–137. [4] K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, San Diego, 1990. [5] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell. 19 (7) (1997) 711–720.