As can be observed in image 1 of Figure 2b, the pre-surgery face images (the images on the first two columns) and the post-surgery images (the image on the third column) show a great amount of skin texture changes. Such differences between the images of the same subject are likely to impact on the face recognition accuracy. A plausible solution is to exploit the face information that are not likely to be affected by plastic surgery. Hence, we exploit the shape of the facial components, i.e., the shape of the eyes, nose (nostrils), eyebrow, and mouth that do not change after plastic surgery procedures. We put forward that this frame of reference serves as a platform for constructing robust and efficient feature descriptors for recognizing surgically altered face images. Under these assumptions, we utilize edge information, which are dependent on the shapes of the significant facial components of the face to address the intra-subject variations due to plastic surgery procedures. The basic idea of the proposed edge-based Gabor face representation approach is aimed at mitigating the intra-subject variations induced by plastic surgery procedures. This is achieved via computing the edge gradient magnitude of the illumination-normalized image. Applying Gabor wavelet on the resultant edge gradient magnitude image accentuates on the uniqueness of significant facial components, which enlarges the discrimination margin between different person face images. These processes are discussed below.
3.1 Edge gradient magnitude computation
Let the grayscale version of the illumination-normalized image Ψ
k
(c) be denoted as Ψ(c). The edge information g(c) of the image Ψ(c) is obtained via the computation of the gradient magnitude of the image; thus [46],
(12)
where and denote partial derivatives, with S as smoothening Sobel filter function.
The false edges in the gradient magnitude image g(c) are substantially reduced when the rgbGE normalization technique is employed. This can be observed in Figure 3a.In Figure 3, the gradient of the rgbGE normalized face images (three images of a subject) is compared with the original image without correction and with various illumination normalization methods such as LT, HE, and GC. It can be seen from the figure that the gradient of the rgbGE face images shows less facial appearance differences in comparison to the other methods. In subsequent subsections, the Gabor encoding process is given in detail.
3.2 Gabor wavelets
Gabor wavelets (kernels and filters) have proven useful in pattern representation due to their computational properties and biological relevance [3, 7, 11, 19]. It is a powerful tool that provides spatial domain and frequency domain information on an object.
The Gabor kernels can be expressed by [47]
(13)
where μ and ν define the orientation and scale of the Gabor kernels, respectively, c = (x, y), ∥ . ∥ denotes the norm operator. The term lμ,ν is defined as [11]
(14)
where and φ
μ
= πμ/8. lmax is the maximum frequency, sf is the spacing factor between kernels in the frequency domain [47], and σ is a control parameter for the Gaussian function.
The family of self-similar Gabor kernels in (13) is generated from a mother wavelet by selecting different center frequencies (scales) and orientations. In most cases, the Gabor wavelets at five scales ν ∈ {0, …, 4} and eight orientations μ ∈ {0, …, 7} are used [11, 19]. This paper uses Gabor kernels at five scales and eight orientations with the following parameters: σ = 2π, lmax = π/2, [11, 19] as shown in Figure 4. The edge image g(c) is convolved with a family of Gabor kernels at five scales and eight orientations; thus,
(15)
where ∗ denotes the convolution operator, and Ομ,ν(c) is the corresponding convolution result at different scales ν and orientations μ.
Applying the convolution theorem, each Ομ,ν(c) from (15) can be derived via the fast Fourier transform (FFT) [11]:
(16)
where ℑ and ℑ−1 denote the Fourier transform and its inverse, respectively.
The Gabor wavelet representation of the edge image g(c) is shown in Figure 5, where only the magnitude responses of Ομ,ν(c) is used to construct the Gabor feature. Having computed Ομ,ν(c), the augmented feature vector, namely the edge-based Gabor magnitude (EGM) feature matrix Ζ(p), is obtained by concatenating each Ομ,ν(c) already downsampled by a factor p, where p = 64 becomes and normalized to zero mean and unit variance. By so doing, the augmented EGM feature matrix Ζ(p) encompasses every possible orientation selectivity, spatial locality, and frequency of the representation result; thus,
(17)
where T is the transpose operator, are the respective J × K downsampled image matrices with orientation μ and scale ν, and Z
rq
are the elements of the R × Q EGM feature matrix. The procedure for obtaining the EGM feature is clearly illustrated in Figure 6.
3.3 Dimensionality reduction and discriminant analysis
The EGM features are of high dimensional space, such that Ζ(p) ∈ RN, where N is the dimensionality of the vector space. To address the dimensionality problem and still retain the discriminating information for identifying a face, we apply the two-stage (PCA + LDA) approach [14, 48]. Each same person face is defined as belonging to a class. Let ω1, ω2, ⋯, ω
L
and N1, N1, ⋯, N
L
denote the classes and the number of images within each class, respectively. Let M1, M1, ⋯, M
L
and M be the mean values of the classes and the grand mean value. The within-class scatter matrix S
ω
and the between-class scatter matrix Sb are defined as [14, 48]
(18)
(19)
where Y(p) is the most expressive feature of the original data Ζ(p) obtained with a PCA step so that LDA is implemented in the PCA subspace [14]. P(Ω
i
) is the probability of the i th class, and L denotes the number of classes.
The LDA derives a projection matrix A that maximizes the Fisher's discriminant criterion:
(20)
The Fisher's discriminant criterion is maximized when A consists of the eigenvectors of the matrix [48].
(21)
where A and Δ are the eigenvector and eigenvalue matrices of , respectively. The two-stage (PCA + LDA) dimensionality reduction approach is employed to maximize the between-class variations and minimize the within-class variations of the projected face subspace.For validation purpose, the face recognition performance of Gabor, i.e., with the original gray-level face images, and the EGM, i.e., with the gradient magnitude face images, are shown in Figure 7.
It can be observed that the use of gradient magnitude image improved the performance of the Gabor descriptor significantly compared to using the original gray-level face images. At this point, it is important to note that the illumination normalization technique that can be used with the EGM include any of the existing normalization techniques discussed in this work. For simplicity, and hence forth, we use the acronym EGM-rgbGE to represent the EGM that employs the rgbGE illumination normalization technique, EGM-HE to represent the EGM that employs the HE illumination normalization technique, and EGM-GC to represent the EGM that employs the GC illumination normalization technique.