1 / 25

Face Image Retrieval by Shape Manipulation

CVPR 2011 Brandon Smith (University of Wisconsin-Madison) Shengqi Zhu (University of Wisconsin-Madison) Li Zhang. Face Image Retrieval by Shape Manipulation. Outline. Introduction Related Work System Overview Technical Details Experiments. Introduction.

amena
Download Presentation

Face Image Retrieval by Shape Manipulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CVPR 2011 Brandon Smith (University of Wisconsin-Madison) Shengqi Zhu (University of Wisconsin-Madison) Li Zhang Face Image Retrieval by Shape Manipulation

  2. Outline • Introduction • Related Work • System Overview • Technical Details • Experiments

  3. Introduction the user drags the tip of the nose leftward to search for similar faces with leftward pose in a sizable database

  4. Introduction • Face Image Retrieval • example based • attribute based • Disadvantage • Shape manipulation based

  5. Related Work • Being john malkovich. In ECCV, 2010 • “ Ever wanted to be someone else? Now --- you can.”

  6. Related Work • Face poser: Interactive modeling of 3d facial expressions using facial priors. In SIGGRAPH, 2010 • generate a new 3D model from example models

  7. Related Work • A generative shape regularization model for robust face alignment. In ECCV, 2008

  8. Tensor Algebra Given data matrix X ( 2-dimensional) PCA decomposes X The scalar version of Eq.(1) is The 3-dimensional array

  9. the matrix form of tensor product

  10. System Overview consists of the following four components • Face database construction • Tensor model training • Tensor coefficient recovery • Search by tensor coefficient comparison

  11. System Overview • Face database construction • construct a sizable database of aligned faces that exhibit a wide range of pose and facial expression variation • a novel confidence score to filter out poor alignment results

  12. System Overview • Tensor model training • Form a 4-dimensional data tensor X, with each of the four dimensions • vertex • Pose • expression • Identity • a new and simple iterative algorithm to achieve this decomposition in the presence of missing data

  13. System Overview • Tensor coefficient recovery • Using the Z andUvert, we associate each aligned face in the databasewith three coefficient vectors, cpose, cexpr, and ciden for pose, expression, and identity • Search by tensor coefficient comparison

  14. Technical Details • Tensor Decomposition with Missing Data • Even highly structured datasets like Multi - PIE have missing faces • minimizing error function • separate a data tensor X into two parts • estimate Xmissing, Z, Uvert, Upose, Uexpr, and Uiden

  15. The error function • initialize Xmissing • the mean average of Xknown ( pose , expression , and identity ) • Iterate between the following steps to minimize Eq. (9) • Fix Xmissing and optimize Z, Uvert, Upose, Uexpr, and Uiden • Fix all the U’s and optimize Xmissing and Z

  16. Tensor Coefficient as Facial Feature Vector • Algorithm • Input : Face shape • Output : three coefficient vectors of the face ( pose, expression, and identity ) • Express face shape vector • Z and Uvert are estimated from

  17. Estimate all of the c’s from fknownby a function as follows • the rows in Uvert • the errors by Eq. (9) • Iteratively the Eq.(13) • hold two of them constant and update the remaining one until Eq. (13) decreases by 10^-6

  18. Searching for Faces Using Tensor Coefficients • Using cpose,cexpr, and ciden enables a user to search images by shape manipulation • Ambiguity • Example : drag the corner of mouth

  19. Face Alignment Confidence Measure • alignment confidence score • dnis the average distance from point n to its neighbors • found by the alignment algorithm [4] • Sn range [ 0 , 1 ] • 1 maximum confidence • 0 no confidence

  20. Filter the point confidence scores • g(sn) is a Gaussian filter of the confidencescores • alignment confidence for a face shape • N is the number of landmarks

  21. Experiments

  22. Constructing a Database of Aligned Faces • Dataset • Public Figures (PubFig) dataset • 50,000 images • Training set • Multi-PIE dataset • 1470face • 5 poses • 6 expressions • we supplied 962 faces • not include faces with both non-frontal pose and non-neutral expression

  23. Face Retrieval Performance • users only need to edit one or a few face points for our system to find desired faces

  24. THANK YOU !

More Related