- Open Access
Face recognition using nonparametric-weighted Fisherfaces
© Li et al; licensee Springer. 2012
Received: 9 May 2011
Accepted: 26 April 2012
Published: 26 April 2012
This study presents an appearance-based face recognition scheme called the nonparametric-weighted Fisherfaces (NW-Fisherfaces). Pixels in a facial image are considered as coordinates in a high-dimensional space and are transformed into a face subspace for analysis by using nonparametric-weighted feature extraction (NWFE). According to previous studies of hyperspectral image classification, NWFE is a powerful tool for extracting hyperspectral image features. The Fisherfaces method maximizes the ratio of between-class scatter to that of within-class scatter. In this study, the proposed NW-Fisherfaces weighted the between-class scatter to emphasize the boundary structure of the transformed face subspace and, therefore, enhances the separability for different persons' face. The proposed NW-Fisherfaces was compared with Orthogonal Laplacianfaces, Eigenfaces, Fisherfaces, direct linear discriminant analysis, and null space linear discriminant analysis methods for tests on five facial databases. Experimental results showed that the proposed approach outperforms other feature extraction methods for most databases.
Face representation is important in recognizing face in many applications such as database matching, security systems, face indexing on web pictures, and human-computer interfaces. The appearance-based method is one of the well-studied techniques for face representation [1, 2]. Two purposes of the appearance-based method are reducing dimensionality and increasing discriminability of extracted features. Hence, a good feature extraction method helps recognize face in a highly discriminative subspace with low dimensionality.
Two of the most classical feature extraction techniques for this purpose are the Eigenfaces and Fisherfaces methods. Eigenfaces  applies principal component analysis (PCA) to transform facial data to the linear subspace spanned by coordinates that maximize the total scatter across all classes. Unlike the Eigenfaces method, which is unsupervised, the Fisherfaces method is supervised. Fisherfaces applies linear discriminant analysis (LDA) to transform data into directions with optimal discriminability. LDA searches for coordinates that separate data of different classes and draw data of the same class close. However, both Eigenfaces and Fisherfaces see only the global Euclidean structure, which may lose some discriminability contained in other hidden structures.
To discover local structure, He et al.  and Cai et al.  proposed the Laplacianfaces method  and its orthogonal form, which is referred to as O-Laplacianfaces . The Laplacianfaces algorithm is based on the locality preserving projection (LPP) algorithm, which aims at finding a linear approximation to the eigenfunctions of the Laplace Beltrami operator on the face manifold. Han et al.  proposed the eigenvector-weighting function based on graph embedding framework.
Recently, many LDA-based methods have been proposed to embed manifold structure into the facial feature extraction process [6–13]. Park and Savvides  proposed a multifactor extension of LDA. Na et al.  proposed the linear boundary discriminant analysis, which increases class separability by reflecting different significances of nonboundary and boundary patterns.
There are several drawbacks in LDA. First, it suffers from the singularity problem, which makes it hard to perform. Second, LDA has the distribution assumption which may make it fail in applications where the distribution is more complex than Gaussian. Third, LDA cannot determine the optimal dimensionality for discriminant analysis, which is an important issue but has often been neglected previously. Fourth, applying LDA may encounter the so-called small sample size problem (SSSP) .
However, the classical LDA formulation requires the nonsingularity of the scatter matrices involved. For undersampled problems, where the data dimensionality is much larger than the sample size, all scatter matrices are singular and classical LDA fails. Many extensions, including null space LDA (N-LDA)  and orthogonal LDA (OLDA), have been proposed in the past to overcome this problem. N-LDA aims to maximize the between-class distance in the null space of the within-class scatter matrix, while OLDA computes a set of orthogonal discriminant vectors via the simultaneous diagonalization of the scatter matrices.
Direct linear discriminant analysis (D-LDA)  is an extension of LDA to deal with SSSP. D-LDA does not use the information inside the null space, as some discriminative information may be lost. D-LDA will be equivalent to N-LDA and LDA in high-dimensional data and small sample size.
In this study, we propose an appearance-based face recognition scheme called nonparametric-weighted Fisherfaces (NW-Fisherfaces). The NW-Fisherfaces approach is a derivative of the nonparametric-weighted feature extraction (NWFE) , which performs well in the studies of hyperspectral image classification [18, 19]. The proposed NW-Fisherfaces method weights the between-class scatter to emphasize the boundary structure of the transformed face subspace and, therefore, enhances the face recognition discriminability. The proposed approach is compared with O-Laplacianfaces, Eigenfaces, Fisherfaces, N-LDA, and D-LDA methods for tests on five face databases. Experimental results show that the proposed approach gains the least error rates in low-dimensional subspaces for most databases.
The rest of this article is organized as follows. Section 2 gives a brief review of related studies. Section 3 introduces the NW-Fisherfaces algorithm. Section 4 presents the experimental results on face recognition. In Section 5, we draw some conclusions and provide some ideas for future research.
2. Related study
Linear feature extraction methods can reduce excessive dimensionality of image data with simple computation. In essence, linear methods project high-dimensional data to low-dimensional subspace.
where WPCA is the set of n-dimensional eigenvectors of ST corresponding to the m largest eigenvalues.
where μ is the mean of all samples, N i is the number of samples in class i, μ i is the average of class i, and is the j th sample in class i. Sw is the within-class scatter matrix. Sb is the between-class scatter matrix. WLDA is the set of generalized eigenvectors of (Sw)-1Sb corresponding to the m largest generalized eigenvalues.
where St is population scatter matrix.
In this new LDA method, they proved that the most expressive vectors derived in the null space of the within-class scatter matrix using PCA are equal to the optimal discriminant vectors derived in the original space using LDA. This method is more efficient, accurate, and stable to calculate the most discriminant projection vectors based on the modified Fisher's criterion (7). This process starts by calculating the projection vectors in the null space of the within-class scatter matrix Sw. This null space can be spanned by those eigenvectors corresponding to the set of zero eigenvalues of Sw. If this subspace does not exist, i.e., Sw is nonsingular, then St is also nonsingular. Under these circumstances, we choose those eigenvectors corresponding to the set of the largest eigenvalues of the matrix (Sb + Sw)-1S b as the most discriminant vector set; otherwise, the SSSP will occur.
where cos(·) denotes the cosine distance measure, i and j denote sample indices, and k and l denote classes. The applied S preserves the locality depending on the cosine distance measure and ensures preservation only for within-class face by setting the between-class weights as 0.
3. Methodology: NW-Fisherfaces
where N i is the training sample size of class i, is the k th sample of class i, denotes the weighted mean corresponding to for class j, and dist(x, y) is the distance measured from x to y. The closer and are, the larger the weight is. The sum of for class i is one. The weight for computing weighted means is a function of and . The closer and are, the larger is. The sum of for is one.
where diag(.) denotes the diagonal part of a matrix.
PCA projection: Face images are projected into the PCA subspace by throwing away the components corresponding to zero eigenvalue. WPCA denotes the transformation matrix of PCA projection. The projected components are statistically uncorrelated and the rank of the projected data matrix is equal to the data dimensionality. This study applied the PCA projection method proposed in [5, 25] to prevent the singularity of Sw due to the simple computation and fair comparison with Fisherfaces and O-Laplacianfaces. However, throwing the dimensionalities corresponding to zero eigenvalue may lose important discriminant information . For further applying LDA-based methods to practical applications, an advanced regularization method proposed in  is suggested.
Compute the distances between each pair of samples and form the distance matrix.
Compute with the distance matrix.
Use to compute the weighted means .
Compute the scatter matrix weight .
Compute and the regularized .
Compute WNWFE = [w1,..., w m ] as the eigenvectors of corresponding to the m largest eigenvalues.
- (8)Compute NWFE embedding as follows:
where W is the transformation matrix and the column vectors of W are the so-called NW-Fisherfaces.
4. Experimental results
The performance of the proposed NW-Fisherfaces method was compared with the three most popular linear methods in face recognition: Eigenfaces , Fisherfaces , and O-Laplacianfaces . Three face databases were tested: Yale database, Olivetti Research Laboratory (ORL) database, and the PIE (pose, illumination, and expression) database from CMU . This study applied the same preprocessing in  to locate face. Gray level images were manually aligned, cropped, and re-sized to 32 × 32 pixels. Each image was represented by a 1,024-dimensional vector. For simplicity, the k nearest-neighbor (k-nn) classifier, where k = 1, was applied in all experiments. Recognition processes were as follows: face subspace was calculated from training samples; new testing face images were projected into calculated subspace; and new facial images were identified by the 1-nearest neighbor classifier.
4.1. ORL database
Performance Comparisons on the ORL Database
4.2. Yale database
Performance comparisons on the Yale database
4.3. PIE database
Performance comparisons on the PIE database
There is no result for N-LDA for PIE database after 10 Train. Because the sample number in training set is larger than the dimension of feature, there was no null space for within scatter matrix Sw.
4.4. PIE_Small database
Performance comparisons on the PIE_Small database
4.5. AR database
In order to check the capability of invariance to lighting condition and face orientation, which have been better solved by 3D deformation approaches. We used AR face database for our proposed method and we found that it is giving better result compare to other method which has been proposed previously.
Performance comparisons on the AR database
The pictures were taken at the CVC under strictly controlled conditions. No restrictions on wear (clothes, glasses, etc.), make-up, hair style, etc. were imposed to participants. Each person participated in two sessions, separated by 2 weeks (14 days) time. The same pictures were taken in both sessions.
In this face database, there are totally 13 expressions of each person. The expressions are as follows: Neutral expression, Smile, Anger, Scream, Left light on, Right light on, All side lights on, Wearing sun glasses, Wearing sun glasses and left light on, Wearing sun glasses and right light on, Wearing scarf, Wearing scarf and left light on, Wearing scarf and right light on 14 to 26: second session (same conditions as 1 to 13).
5. Conclusions and future works
The proposed NW-Fisherfaces consistently outperforms the Eigenfaces, Fisherfaces, D-LDA, and N-LDA methods.
This study applied a nonparametric feature extraction method into the scheme of appearance-based face recognition.
The proposed NW-Fisherfaces method weights the between-class scatter to emphasize boundary structure of the transformed face subspace and, therefore, enhances the discriminability of face recognition.
For practical applications, the computational load will depend on the dimensionality of the trained linear projection matrix. In this study, experimental results show that the proposed method can reach its lowest error rate with low dimensionality. Hence, the NW-Fisherfaces method is practical for real-world face recognition due to the low dimensionality requirement.
5.2. Future works
The supervised OLPP weights the scatter matrix to preserve the locality of within class face. This weighting concept may enhance the within-class scatter of LDA and other LDA-based methods such as NDA and NWFE.
Linear feature extraction methods measure and optimize closeness between samples depending on Euclidean distance. However, Euclidean distance is basically light variant. Variance caused by lighting should be reduced before using linear feature extraction methods. Several solutions to reduce light variances of face images are proposed:
Mapping face images into the same intensity distribution by simple preprocessing such as histogram specification.
Transforming images into frequency domain by Fourier-based methods such as Gabor wavelets.
The performance of NW-Fisherfaces in nonlinear feature space, such as kernel Hilbert space, can be further evaluated.
The authors would like to thank the editors and reviewers for their helpful comments. They would also like to thank Dr. Li-Wei Ko, Department of Electrical and Control Engineering, National Chiao-Tung University, Taiwan, for their support in writing this article, and Dr. Deng Cai, for his essential contribution and providing a complete framework on face recognition.
- Han PY, Jin ATB, Siong LH: Eigenvector weighting function in face recognition. Discrete Dyn Nat Soc 2011: 521935. doi:10.1155/2011/521935Google Scholar
- Murase H, Nayar SK: Visual learning and recognition of 3-D objects from appearance. Int J Comput Vis 1995, 14: 5-24. 10.1007/BF01421486View ArticleGoogle Scholar
- Turk M, Pentland AP: Face recognition using Eigenfaces. Presented at the IEEE Conf Computer Vision and Pattern Recognition Maui, HI 1991, 586-591.Google Scholar
- He X, Yan S, Hu Y, Niyogi P, Zhang H-J: Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 2005, 27(3):328-340.View ArticleGoogle Scholar
- Cai D, He X, Han J, Zhang H-J: Orthogonal Laplacianfaces for face recognition. IEEE Trans Image Process 2006, 15(11):3608-3614.View ArticleGoogle Scholar
- Park SW, Savvides M: a multifactor extension of linear discriminant analysis for face recognition under varying pose and illumination. EURASIP J Adv Signal Process 2010: 158395. doi:10.1155/2010/158395Google Scholar
- Na JH, Park MS, Choi JY: Linear boundary discriminant analysis. Pattern Recogn 2010, 43(3):929-936. 10.1016/j.patcog.2009.09.015View ArticleGoogle Scholar
- Sanayha W, Rangsanseri Y: Weighted LDA image projection technique for face recognition. IEICE Trans Fund 2009, E92.A(9):2257-2265. 10.1587/transfun.E92.A.2257View ArticleGoogle Scholar
- Sugiyama M: Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 2007, 8: 1027-1061.Google Scholar
- Liu C, Wechsler H: Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans Image Process 2002, 11(4):467-476. 10.1109/TIP.2002.999679View ArticleGoogle Scholar
- Loog M, Duin RPW: Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion. IEEE Trans Pattern Anal Mach Intell 2004, 26(6):732-739. 10.1109/TPAMI.2004.13View ArticleGoogle Scholar
- Liu Q, Lu H, Ma S: Improving kernel Fisher discriminant analysis for face recognition. IEEE Trans Circ Syst Video Technol 2004, 14(1):42-49. 10.1109/TCSVT.2003.818352View ArticleGoogle Scholar
- Yan S, Hu Y, Xu D, Zhang H-J, Zhang B, Cheng Q: Nonlinear discriminant analysis on embedded manifold. IEEE Trans Circ Syst Video Technol 2007, 17(4):468-477.View ArticleGoogle Scholar
- Fukunaga K: Introduction to Statistical Pattern Recognition. Academic Press, New York; 1990.Google Scholar
- Chen L-F, Mark Liao H-Y, Ko M-T, Lin J-C, Yu G-J: A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognition 2000, 13(10):1713-1726.View ArticleGoogle Scholar
- Wu X-J, Kittler J, Yang J-Y, Messer K, Wang S: A new direct LDA (D-LDA) algorithm for feature extraction in face recognition. Proceedings of the 17th International Conference on Pattern Recognition 2004 ICPR 2004 2004, 4: 545-548.Google Scholar
- Kuo B-C, Landgrebe DA: Nonparametric weighted feature extraction for classification. IEEE Trans Geosci Remote Sens 2004, 42(5):1096-1105.View ArticleGoogle Scholar
- Benediktsson JA, Palmason JA, Sveinsson JR: Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Trans Geosci Remote Sens 2005, 43(3):480-491.View ArticleGoogle Scholar
- Kuo B-C, Chang K-Y: Feature extractions for small sample size classification problem. IEEE Trans Geosci Remote Sens 2007, 45(3):756-764.View ArticleGoogle Scholar
- Belhumeur PN, Hepanha JP, Kriegman DJ: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 1997, 19(7):711-720. 10.1109/34.598228View ArticleGoogle Scholar
- Yu H, Yang J: A direct LDA algorithm for high-dimensional data-with application to face recognition. Pattern Recogn 2001, 34: 2067-2070. 10.1016/S0031-3203(00)00162-XView ArticleGoogle Scholar
- Liu K, Cheng YQ, Yang JY: A generalized optimal set of discriminant vectors. Pattern Recogn 1992, 25(1):731-739.View ArticleGoogle Scholar
- Lu JW, Plataniotis KN, Venetsanopoulos AN: Face recognition using LDA-based algorithms. IEEE Trans Neural Netw 2003, 14(1):195-200. 10.1109/TNN.2002.806647View ArticleGoogle Scholar
- Sim T, Baker S, Bsat M: The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 2003, 25(12):1615-1618. 10.1109/TPAMI.2003.1251154View ArticleGoogle Scholar
- Martinez AM, Kak AC: PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 2001, 23(2):228-233. 10.1109/34.908974View ArticleGoogle Scholar
- Mandal B, Jiang X, Eng H-L, Kot A: Prediction of eigenvalues and regularization of Eigenfeatures for human face verification. Pattern Recogn Lett 2010, 31(8):717-724. 10.1016/j.patrec.2009.10.006View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.