Advancing state-of-the-art 3D human facial recognition

Automated human face recognition has numerous applications in areas of security and surveillance, database retrieval, and human-computer interaction. Over two decades of research have resulted in successful techniques for recognizing color/intensity 2D frontal facial images.¹ However, performance declines severely with variations in pose, expression, and ambient illumination.² Achieving robust and accurate automatic face recognition remains an important and open problem.

Recently, researchers have proposed the use of 3D models.³ In addition to providing explicit information about the shape of the face, these models can easily correct for pose by rigid rotation in 3D space, and are scale and illumination invariant. However, successful 3D facial recognition techniques² based on rigid surface matching suffer from an exceptionally high computational cost that makes them unsuitable for real-time operation. Existing 3D techniques are also unable to handle changes in facial expression. We have attempted to resolve some of these issues.

Features for face recognition are numerical quantities that vary considerably between individuals yet remain constant for different instances of the same individual. To date, 3D recognition algorithms that are based on local features have not embodied a sound understanding of discriminatory facial structural characteristics. As a starting point, we extensively investigated the existing literature on anthropometric facial proportions.⁴ We identified measurements reported to be highly diverse across different age and gender cohorts and ethnic groups. We employed these to design novel and effective algorithms.⁵

Specifically, we used 3D euclidean and along-the-surface geodesic distances between 25 manually located facial fiducial points (i.e., loci of interest, see Figure 1), associated with the highly variable anthropometric measurements, as features for face recognition.⁵ We studied geodesic distances because a recent study⁶ suggests that changes in expression may be modeled as isometric deformations of the face, under which geodesic distances between pairs of points on the facial surface remain constant. We employed stepwise linear discriminant analysis (LDA) to select a subset of the most discriminatory features and followed it with an LDA classifier. With a gallery set containing 1 image of each of 105 subjects and a probe set containing 663 images of the same subjects, our algorithm produced an equal error rate of 1.4% and a rank 1 recognition rate of 98.6%. It significantly outperformed the existing baseline algorithms based on principal component analysis (PCA) and LDA applied to face range images (see Figure 2). Our proposed technique was also robust with respect to changes in facial expressions.

Figure 1. (a) The facial fiducial points associated with discriminatory facial anthropometric measurements on a color image. (b) These points on a facial 3D depth map image (reproduced from Gupta et al.⁵).

Figure 2. Cumulative match characteristic curves for 3D face recognition algorithms based on distances between anthropometric facial fiducial points, and principa component analysis (PCA) and linear discriminant analysis (LDA) applied to range images.

For rigid facial surface matching, the existing approaches rigidly and closely align a pair of surfaces using an optimization procedure, then compare them by means of a distance metric. These techniques suffer from the very high computation cost for calculating the point-to-closest-point mean squared error (MSE_CP)/Hausdorff distance (HD) metrics and for the iterative alignment procedure. We attempted to accelerate the process by employing the recently developed complex-wavelet structural similarity metric (CW-SSIM)⁷ to match coarsely aligned facial range images.⁸ This index is more computationally efficient than MSE_CP and HD, and is also robust with respect to small alignment errors. It thus eliminates the need to finely align 3D faces before matching.

With a database of 360 3D facial images of 12 subjects (30 images per subject), our proposed technique achieved a rank 1 recognition rate of 98.6%. It significantly outperformed facial surface matching based on MSE_CP and HD, both in terms of accuracy and computational cost (see Figure 3). This study also extended the scope of application of CW-SSIM to 3D range images.

Figure 3. Cumulative match characteristic curves for rigid facial surface matching algorithms based on different distance metrics.

To summarize, we systematically identified discriminatory facial structural information from other areas of quantified investigation and employed it to design effective 3D facial recognition algorithms. In the future we will also investigate techniques to automatically locate the associated facial fiducial points. Moreover, we have developed novel techniques for rigid facial surface matching using the CW-SSIM similarity index that are more accurate and computationally efficient than existing techniques.

The authors gratefully acknowledge Advanced Digital Imaging Research, LLC (Friendswood, TX), for funding and 3D face data.