Face Recognition Using Optimized 3D Information from Stereo Images

In this paper we propose a new range-based face recognition for significant improvement in the recognition rate using an optimized stereo acquisition system. The optimized 3D acquisition system consists of an eyes detection algorithm, facial pose direction distinction, and principal component analysis (PCA). The proposed method is carried out in the YCbCr color space in order to detect the face candidate area. To detect the correct face, it acquires the correct distance of the face candidate area and depth information of eyes and mouth. After scaling, the system transfers the pose change according to the distance. The face is finally recognized by the optimized PCA for each area with the facial pose elements detected. Simulation results with face recognition rate of 95.83% (100cm) in the front and 98.3% with the pose change were obtained successfully. Therefore, proposed method can be used to obtain high recognition rate with an appropriate scaling and pose change according to the distance.


Proposed stereo vision system
In order to acquire the distance and depth information, we use a parallel stereo camera as shown in Figure 2. From the stereo camera, we obtain the disparity between left and right images and estimate the distance by a stereo triangulation.

Disparity compensation of stereo images
A block matching algorithm is used to extract the disparity in the stereo images, after applying 3 3 × Gaussian noise smoothing mask.
In general, the block matching algorithm uses the mean absolute difference (MAD) or the mean square difference (MSD) as a criterion. However, the proposed method uses the sum of absolute difference (SAD) to reduce computational complexity as where L I represents the corresponding block of right image, and k represents the disparity between left and right images. In the stereo image matching, the disparity compensation between left and right images should be performed. When a point in the 3D space is projected on left and right images, the virtual line connecting two points is called an epipolar-line [9]. The corresponding blocks of the stereo images are matched on the epipolar-line with the same xcoordinate. The modified block matching algorithm based on 4×4 block is used for fast processing as shown in Figure 3.  The proposed block matching algorithm can remove unnecessary operations and the performance of the proposed block matching algorithm is as good as the one of the global searching algorithm. The process of the proposed algorithm is as following. First, SAD is calculated at each row and then the minimum value of SAD at the corresponding row is obtained as Finally, the minimum SAD of entire image can be obtained as ( By using the camera characteristics as given in Table 1, the distance can be measured as where b represents the width between cameras, f represents the focal length, and x l and x r respectively represent the distances of left and right images. Also, the constant of 86.80×103 represents the effective distance per pixel.
where ' x , ' y represent the position after scaling processing, x s , y s represent the scaling factor, and x , y represent the current position. From the obtained distance in (6), the scaling factor of face image can be calculated as where dist B , dist V , and dist A , and represent the basic distance, the established value by distance, and the obtained distance, respectively.

Range-based pose estimation using optimized 3D information
In order to solve the problem of the low recognition rate due to the uncertainty of size, distance, motion, rotation, and depth, optimized 3D information from stereo images is used. By estimating the position of eyes, the proposed method can estimate the facial size, depth, and pose change, accurately. The result of estimation of facial pose change is shown in Figure 4. In Figure 4, the upper and lower images respectively represent the right image and the left image of frontal face. In Figure 5, the range of 9 directions for face images is defined to estimate the accurate facial direction and position of stereo images.

Pose estimation and face recognition
Face recognition rate is sensitive to illumination change, pose and expression change, and resolution of image. In order to increase the recognition rate under such conditions, we should consider the pose change as well as the frontal face image. The recognition rate can be increased by the 3D pose information as presented in Figure 5. In order to detect face region and estimate face elements, the multi-layered relative intensity map based on the face characteristics is used, which can provide better result than the method using only color images. The proposed directional blob template can be determined according to the face size. In detail, to fit for the ratio of the horizontal and vertical length of eyes, the template should be defined so that the length of horizontal axis is longer than that of vertical one as shown in Figure 6 (a   The classified images are trained by PCA algorithm using optimized 3D information component. The block diagram of the proposed optimized PCA algorithm is shown in Figure 8.

Experimental Results
For the experiment, we extracted 50 to 400 stereo pairs of face images of size 320 240. Figure 9 shows the matching result of the left and the right images captured in the distance of 43cm. Composed image shows Figure 9(c) which initializes 20 10 block in Figure 9(a), and is searched in the limited region of Figure 9(b). The disparity can be found in the most left and the top regions as shown in Figure 9(c). Facial pose estimation is performed with 9 directional groups at 100cm by using the proposed system as shown in Figure 10.  Table 2, the recognition rate is compared according to the distance. As shown in the Table  2, the highest recognition rate can be obtained at the reference distance of 100cm. After training 200 stereo images, the recognition rates of the proposed methods were compared to those of the existing methods with respect to 120 test images. The recognition rate of the proposed method based on optimized 3D information is provided in Figure 14. Experiment 1 and 2 respectively used frontal face images and images with various pose change. Figure  14 shows that the recognition rate using the conventional PCA or HMM drops in inverse proportion to the distance. From the experiments, the proposed method can increase the recognition rate.  Figure 14. Recognition rates versus distance comparison for the proposed and various existing methods

Conclusions
This paper proposed a new range-based face detection and recognition method using optimized 3D information from stereo images. The proposed method can significantly improve the recognition rate and is robust against object's size, distance, motion, and depth using the PCA algorithm. The proposed method uses the YCbCr color format for fast, accurate detection of the face region. The proposed method can acquire more robust information against scale and rotation through scaling the detected face image according to the distance change. Experiments were performed in the range of 30~200cm and we could get the recognition rate up to 95.8% according to the scale change. Also, we could get the high recognition rate of 98.3% according to the pose change. Experimental results showed that the proposed method can increase the low recognition rate of the conventional 2Dbased algorithm.