Face Processing System

1. Face Processing System Presented by: Harvest Jang Group meeting Fall 2002

2. Outline • Introduction • System Architecture • Pre-processing • Face detection • Face tracking • Problems • Future work

3. Introduction • An automatic system is needed to provide more human-computer interactive services to improve the life style • Security System • Interactive application/game • Robot visualization • This is a challenging task for integrating different techniques • face detection • face tracking • face recognition

4. System Architecture • Three main components • Pre-processing • Face detection • Face tracking Fig 1: System architecture

5. Pre-processing • Transform to YCrCb color space • Use ellipse color model to locate the fresh color Fig 2: 2D projection in the CrCb subspace (gray dots represent skin color samples and black dots represent non-skin tone color)

6. Pre-processing • Perform morphological operation to reduce noise • Skin segmentation to find face candidates Fig 3: Pre-processing step (a) binary skin mask, (b) original images (c) binary skin mask after morphological operation and (d) Face candidates

7. Comparison on three face detection approach • Motivation for the comparison • Using different infrastructure systems for detecting faces in a large image • Only shows one result pair of detection rate and false alarm rate • Using experimental result to compare • Sparse Network of Winnows (SNoW) • Support Vector Machines (SVM) • Neural Network

8. Comparison on three face detection approach • Sparse Network of Winnows (SNoW) • Primitive features architecture • Winnow update rule Table 1: Winnow update rule

9. x0= 1 x1 w1 w0 x2 w2  xn wn Comparison on three face detection approach • Support Vector Machines (SVM) • Find a hyperplane that leaves the maximum margin between two classes which will have the smallest generalization error • The margin is defined as the sum of the distances of the hyperplane from the closest point of the two classes • Neural Network • Back-propagation method Fig 4: SVM Fig 5: Neural Network

10. Comparison on three face detection approaches • CBCL face database from MIT • Training set (2429 face pattern, 4548 non-face pattern with 19x19 pixel) • Testing set (472 face pattern, 23573 non-face pattern with 19x19 pixel) • To better represent the detectability of each model, ROC curve is used to replace single point of criterion response

11. Comparison on three face detection approaches • Similar signal for • SNoW • Neural Network • SVM model with linear kernel • SVM model with polynomial kernel • much stronger signal • the classification between face and non-face patterns is much better Fig: 6: ROC curves of different face detection method

12. Comparison on three face detection approaches • In our system, success to detect face is more important than processing more windows • Face tracking method to help for tracking the detected face • SVM model with polynomial kernel of second degree is chosen. Table 2: The average processing time of each model for testing 24045 images ten times

13. Face detection • To detect different size of faces, the region is resized to various scales • A 19x19 search window is searching around the re-sized regions • Histogram equalization is performed to the search window • After the first presence of a face at the larger scale, face recognition using SVM classifier will be performed to retrieve the information of that person.

14. Transform to various scale Histogram equalization Apply a 19x19 search window Face detection - Example

15. Face tracking • Why using face tracking • Face detection method has missing rate and false alarm rate • The missing rate and false alarm rate for losing an object will be reduced • The performance speed is much better than face detection method • Conditional Density Propagation – Condensation algorithm • Sampling-based tracking method

16. Face tracking • The posterior probability density at timestep t using a set of N random samples, denoted as with weights • There are three phases to compute the density iteratively at each time step t for the set of random samples to track the movement

17. Face tracking • Selection phase: • The element with high weights in the set has a higher probability to be chosen to the predictive steps • Predictive phase: • The sample set for the new time-step is generated by independent Brownian motion • The weights are approximately from the effective prior density for time-step t+1

18. Face tracking • Measurement (Update) phase • Given in terms of likelihood • The model is expressed as a histogram for face color HUE (HSV color space) • Calculate the measurement where is a normalization factor

19. Face tracking Fig 7: One time step in the Condensation algorithm

20. Face tracking • The color input image is being masked by the binary skin mask • reduce the localization period • converge less sensitive from the background noise • converge to the face boundary much faster.

21. Problems • The face detection rate is still low • The condensation algorithm doesn’t provide verification for the tracking object • Two adjacent object with similar color will cause problem

22. Future work • Improve the accuracy of face detection • Implement the face tracking verification step using face recognition

23. Thank you!