Biological landmark vs quasi-landmarks for 3D face recognition and gender classification

Received Jan 29, 2019 Revised Apr 13, 2019 Accepted Apr 25, 2019 Face recognition and gender classification are vital topics in the field of computer graphic and pattern recognition. We utilized ideas from two growing ideas in computer vision, which are biological landmarks and quasilandmarks (dense mesh) to propose a novel approach to compare their performance in face recognition and gender classification. The experimental work is conducted on FRRGv2 dataset and acquired 98% and 94% face recognition accuracies using the quasi and biological landmarks respectively. The gender classification accuracies are 92% for quasi-landmarks and 90% for biological landmarks.


INTRODUCTION
Nowadays, our surrounding environment became more technically. The devices are getting to be smarter every day. In the world of technology, one of the vital technological features is artificial intelligence. Face recognition, as a biometric authentication technique is an essential application scope of artificial intelligence [1,2]. Its main superiority inherited in high speed throughput when the user does not spend time in data collection process like other biometric techniques such as speaker recognition [3], fingerprint [4] and iris [5]. For example, face recognition system can be installed in a publicplace where faces of people who pass can be recognized and picking out the wanted persons [6]. The main disadvantage of this 2D software is the sensitivity in lighting conditions, which considered as poses and occlusions. In the last few years, due to the robustness of 3-D techniques in illumination variances and pose alteration, using 3D facial surfaces in face recognition has become a crucial research field.
Simultaneously, gender classification is an important problem that can be a valuable primary step into face recognition. Gender is a soft biometric traits. It can combine other biometric traits such as eyes colors, skin color, and so on, to differentiate between humans peers. Many present works on gender classification have used 2-D face images for solving gender classification problem, but 3-D face surface has not so far been used widely for gender classification. Diverse psychology encourages to use 3-D shape information for gender classification [7,8].
The development of a practical and easy intelligent system for automatic face recognition and gender classification is an important subject. In this research, the effectiveness of using biological landmark (certain vertices of face mesh) and quasi-landmarks (regularised face mesh vertices) on face recognition and gender classification system performance has been investigated. This research main novelty is testing the performance of biological landmarks in face recognition and gender classification. Specifically, the main contributions of this paper include: -It tests the performance of Biological landmark for face recognition and gender classification.  -It tests the performance of quasi-landmarks for face recognition and gender classification.  -It compares the Biological landmark and quasi-landmarks reliability for gender classification and face recognition.
-It proposes a simple automatic face recognition and gender classification system using landmarks and principal component analysis (PCA). The rest of this paper is organised as follows; In Section 2, a brief review of the approaches that are related to the proposed method is introduced. Section 3 displays the procedure of the proposed method. In Section 4, the experimental results are introduced. The discussion for the acquired results is presented in Section 5. Finally, Section 6 concludes the methodology and important achievements of the proposed work, and it includes suggestions for future research investigation.

RELATED WORK
Biological landmarks are standard reference significative points on the face, such as the nose tip, inner and outer corners of the eyes, corners of the mouth, and the chin point [9,10]. Landmark localisation is usually localised manually, but numerous research has been achieved on automatic accurate localisation. For example, Ruiz et al. [11] presented an algorithm for automatic localisation of biological landmarks on 3D faces by using the Active Shape Model (ASM) to localise a set of 14 facial biological landmarks.
A series of different measurements such as the Euclidean distance and the angle between biological landmarks are used successfully in finding the differences among individuals faces. The accuracy of these measurements depends dramatically on landmarks localisation and selection. Morecroft [12] performed a research to evaluate the using of biological landmarks for 3D face shape analysis. Aynechi et al. [13] carried out an algorithm representing a face shape by a vector of distances measured between pairs of facial landmarks. They selected the landmarks by an exhaustive search over possible landmarks combinations which offered the best recognition rates. Their research results recorded 27 facial landmarks as essential landmarks for facial comparison and identification.
Surface regularisation is the process of finding the geometrical transformation between two or more surfaces that align the surfaces [14]. It regularises the number of vertices by finding dense correspondences in the 3D meshes, these regularised vertices are called quasi-landmarks. The quasi landmarks are used dramatically in comparing the differences in facial morphologies for various populations and sexes [15,16].
The clear sexual dimorphism of the human face makes face gender classification considerably efficient and fast operation [7]. In spite of, the abundant gender classification research, the task of developing easy to implement, automated and robust face gender classification is still a remote objective. To overcome the limitation items of 2-D image base methods, three-dimensional (3-D) information of human faces is used to perform face gender classification. For example, in O'Toole et al. [8] principal components analysis (PCA) was applied separately to the 3-D mesh and gray level image pixels of 3-D human face image. The results demonstrated the three-dimensional face mesh provided more precise sex classification results than the gray level image pixels. Yuan et al. [17] used the features from four face regions (upper region of the face, the lower region of the face, the nose and the left eye) for gender classification purpose.
A general review on 3D face recognition methods is introduced in [18]. The difficulty in 3D face recognition algorithms' implementation is handling occlusions, missing face parts, and expression. The existing 3D face recognition methods can be categorised into two categories according to the features used in recognition: holistic methods and regionally approaches. In holistic methods, the features from the whole is used for recognition. Drira et al. [19] implemented a method that uses curvatures of the 3D facial surfaces. The method used the radial curves of facial surfaces around the nose tip for recognition. Morphable model-based approaches were introduced in [20,21], this model used effectively for face recognition. In local-region based approaches, local information is extracted from several face regions. For example, Li et al. [22] proposed a robust local facial descriptor contains four types of geometric features. Then, this descriptor is used for recognition purpose.
In spite of the accuracy of these methods in gender classification and face recognition, they suffer from the following problems: complexity, and slow feature extraction and recognition processes. Therefore, proposed a simple and accurate face regnition method is essential. In this paper, biological landmark and the quasi-landmarks are extracted and used as feature descriptor for 3D face recognition and gender classification. This research represents the first research tests the efficiency of biological landmark and the quasi-landmarks in face recognition and gender classification.

PROPOSED ALGORTHIM
In this section, we describe a proposed approach of face recognition and gender classification using the biological landmark or the quasi-landmarks. Figures 1 and 2 display block diagrams of the proposed approach, which are explained below. Figure 1 diagram illustrates the proposed approach for face recognition, while Figure 2 illustrates the gender classification approach. Figure 3 shows the landmarks extraction block contents.

Dataset
We utilized the FRGC v2 database. This data contain 4,007 3D face scans of 466 subjects. The data were collected using a Minolta 910 type laser scanner. The images record different facial expressions (e.g., happiness and surprise). Maurer et al. manually categorised the FRGC v2 into three categories [23] depending on the facial expression. They classified 796 images as small expressions and 742 images as large expressions, while 2469 images classified as the neutral expression. For face recognition experiment, the input to the algorithm is two sets of images gallery and probe faces. Faces in the gallery set represent facial 3D images known to the system, while faces in the probe set represent unknown facial 3D images presented to the system for the recognition. In this work, faces with neutral expression are used. According to the Jack-knife rule 80% of the faces randomly selected for training while the rest 3D faces are used for proposed algorithm testing. Therefore, 1975 images are used for training and 494 3D face images are used for system testing. For gender classification approach, 466 subjects images are used (202 female and 264 male), also according to Jack-knife technique a total of 372 images were used for training and 94 images were used for testing.

Preprocessing
This stage consists of three processes: Clip the facial region, Remove data spikes (noise) and Fill the holes.
-Denoising: The undesirable elements of the scans were removed manually followed by automatic removal of the large spike, which can naturally arise in scanning. The images were further smoothed using the Laplacian filter available in Rapidform, which removes little spikes. Laplacian smoothing is a common procedure (e.g., see [24]). -Hole filling: Large or small holes may appear in 3D face scans. All holes were specified and filled using Rapidform program tools.

Landmarks Extraction
Biological landmarks and quasi landmarks are used as facial features for face recognition and gender classification. Biological landmarks are usually used by physicians and scientists, they are meaningful points that are defined as standard points on the face, such as the nasion, inner and outer canthi of the eyes, enter of the upper lip, center of the lower lip, mouth corners [25].
In this work, 21 facial landmarks represented by a total of 63 (x, y, and z) coordinate values were manually identified and recorded for each 3D facial image. After that, a Generalised Procrustes Algorithm (GPA) is implemented to carry out the rotation, translation and scaling of the faces landmarks. Ordinary Procrustes analysis is a type of statistical shape analysis that aligns faces by minimizing the Procrustes distance metric between their landmarks [26]. The algorithm steps are the following: -Choose a reference face landmarks among the training set instances. -Align all other faces landmarks on the current reference by minimizing the Procrustes distance between the instances. -Determine the mean face of the training set faces 3D landmarks. -Calculate Procrustes distance between the mean face and the reference face. If this distance value is above the specified threshold, set the reference face to mean face and go to step 2. Figure 5 illustrates the biological face landmarks that used in this research. Figure 6 shows the effect of applying GPA on the landmarks coordinates, the red points is the landmarks coordinates before applying the GPA, while the blue point after applying this algorthim. Quasi landmarks are acquired by superimposing all images on a reference face and then regularising all the images to ensure that the resulting meshes have exactly the number of reference face vertices. Specifically, the following was implemented: -Rigid registration. The idea was to use one of dataset faces as facial template to crop the facial images of interest. The template was brought into the vicinity of the target surface, by repositioning the floating surface, using iterative closest point (ICP) algorithm [27].

4073
-Non-rigid registration. Surfaces of different subjects are matched to each other using a non-rigid Thin Plate Spline (TPS) [28]. -Regularisation. An iterative procedure was used to assign each floating point of the template to a corresponding point of the target image by applying the distance weighted k-nearest neighbor rule [29]. As a result, all faces were cropped and converted to uniformly distributed meshes. Figure 5. The biological face landmarks Figure 6. The GPA effect on landmarks coordinates

Principle Component Analysis (PCA)
PCA is a successful method progressed for 2D or 3D face recognition. In this algorithm, eigen decomposition of the 2D matrix of facial images features is performed to obtain a set of orthogonal basis vectors. These vectors maximise the variance of facial image dataset. Therefore, the PCA is used to reduce the dimensionality of dataset and represents face images into their principal components. For face recognition, the top eigenvectors are known as Eigenfaces. These eigenvectors account the most variation in the data [30]. All gallery and probe facial images are projected onto the principle space and the eigenvectors is used as training and testing elements in face recognition and gender classification system.

Machine Learning techniques
Automatic face recognition and gender classification are pattern recognition problems, which are not easy to solve problems because their non-linearity. Especially, when face recognition and gender classification applied in high dimensional and online dataset. Therefore, using accurate machine learning algorithms are essential for face recognition and gender classification problems.
The K-Nearest Neighbour method (KNN) is a supervised learning algorithm. In this algorithm, the tested instance is classified according to K-nearest neighbor categories. This method is used markedly in numerous computer vision and image processing applications. Support Vector Machine (SVM) are also popular supervised learning algorithm and it is used for solving different machine learning problems. SVM classifier tries to find the optimal results by maximizing the margin between the training and testing instances [30]. Thus, in this research, these two machine learning algorithms are used for face recognition and gender classification.

RESULTS
In the following, we present the implemented experiments and their results. We also perform a comparative review with the state-of-the-art related research results. The FRGCv2 dataset have 4007 3D face images, 60% of the faces have neutral expression. In addition to that the subjects' gender in this dataset are divided into 57% male and 43% female. The face recognition task is multiclass classification, therefore One-Against-All (1AA) techniques are used. In a 1AA approach the classification result is determined for each class by discriminating that class against the remaining (N-1) classes. To access the performance of the recognition and gender classification three measures are used accuracy, sensitivity, and specificity [32]. Table 1 records the recognition accuracy, sensitivity, and specificity of the both types of landmarking (biological landmark and quasi-landmarks), while Table 2 records the gender classification accuracy, sensitivity, and specificity of the same types of landmarks.

DISCUSION
The experimental work in the present study aims to determine which facial landmarks were the most effective features in automatic face recognition and gender classification. For face recognition, the best performance was achieved when the quasi-landmarks are used, as illustrated in Table 1, but the biological landmarks accuracy in face recognition was not far from the quasi-landmarks accuracy. The same observation is reached from analysing the gender classification results in Table 2. That means the biological landmarks can be used as features for accurate and high-performance face recognition and gender classification system. Table 1 and Table 2 results show the performance of SVM classifier is better than KNN classifier in face recognition and gender classification. As shown in Table 3, the proposed approach for face recognition outperforms the state-of-the-art work, which conducted on the same dataset. Table 4 records gender classification accuracies compared to earlier methods implemented on FRGCV2 and FRGCV1 datasets Table 3. Comparison with state of the art methods on FRGCV2 Methods Accuracy F. Daniyal [33] 97.2% Kakadiaris [34] 97.2% D. Huang [35] 97% Cook [36] 92.9%

CONCLUSION
This paper proposes a novel comparison between the quasi and biological landmarks for an effective gender classification and face recognition system. Twenty-one biological landmarks can be used as features for high-performance gender classification and face recognition system. We can notice that the performance results of our 3D face recognition and gender classification system, outperform the reordered results of many other published researches on FRGCV2 data set. In addition to that our system is easier to implement than other states of the art face recognition and gender classification systems. In future, we are planning to extend our approach to person ethnic classification. In this research 21 landmarks are used, testing the performance of different numbers of biological landmarks in face recognition and gender classification, which will also be considered in future work.