1 / 26

3D Shape Retrieval from Silhouette Images

3D Shape Retrieval from Silhouette Images. Nat Duca. Overview (1). Input:. Output:. Edges:. Overview (2). Are silhouettes good descriptions of shape? One may not be… but a few are Can you see 3D structure in the shadow of a hand in the projector?

daveigh
Download Presentation

3D Shape Retrieval from Silhouette Images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 3D Shape Retrieval from Silhouette Images Nat Duca

  2. Overview (1) Input: Output: Edges:

  3. Overview (2) • Are silhouettes good descriptions of shape? • One may not be… but a few are • Can you see 3D structure in the shadow of a hand in the projector? • Silhouettes are easily entered queries… draw the contour of your shape • Also, they can be derived from an arbitrary sketch or image… or shadows, come to think of it

  4. Approach Overview • Sea of cameras: produce multiple views of an object • A silhouette contour of a closed object is a circular function: can be made rotation-invariant • Multiple silhouettes can be used to improve the accuracy of a query (i.e. get me the object that has these two silhouettes)

  5. Sea of cameras • The “sea” of cameras can be sampled over a hemisphere because silhouettes will be the same (mod flip) • Regular distribution on a sphere over samples… • Ideal case: random distribution of K camera points over a hemisphere that changes for each model • My version: custom placed locations of 20 points on a hemisphere that is constant across all shapes

  6. Contour extraction (1) • Most practical shapes will have: • Sampling artifacts in the silhouette image • Interior contours (i.e. no one dominant contour) silhouette edges

  7. Contour extraction (2) • Perform morphological closing of silhouette image with a disk structuring element • Use a laplacian-of-gaussian edge detector to extract contour pixels

  8. Contour extraction (3) • Using a maze walking algorithm, find a closed loop through the contour pixels to produce a single closed group of pixels • This is done for all connected components • I bet we could use all of the components to do partial matching on this shape • … but for now, we chose the largest contour that results

  9. Silhouette Parameterization • Goals: rotation, scale, translation invariance • RDF (radial distance function) • f(c) = r where r is maximal object extent in direction c from centroid • As in spherical harmonics • Arc length parameterization: • f(c) = r + k(s)i parameterization about centroid • Where k(s) is the stretch required to fit the curve to the surface

  10. Matching Metric • Let dist(s1, s2) be the sum of squared distances between the descriptor for two contours (“samples”) • Option 1: best sample from query and model • Let dist(mi,q1…K) for some model mi and K samples q1…j be defined as • The sample sd in model mi that is closest to any qi • Option 2: all samples in query, best in model • Sum over all k of qi • The sample sd in model mi that is closest to qi

  11. System Details (1) • I start with Princeton Shape Benchmark geometry and classifications • Silhouette sea is generated by a glut app in Linux using depth buffer readback • Images are copied to a machine with Matlab where morphological closing and edge detection is done • … and then copied back… grr… • Useful stat: 622 models yields 1.5 GB of images, but only a 7.5mb database

  12. System Details (2) • Edge images are processed by a maze search algorithm to find the largest closed contour • Contours are sampled evenly to 200 points using Gnu Scientific Library • The parameterization is FFT’d with FFTw • The resulting descriptors are stored in a binary database • We can set an epsilon value that allows the same sample to be appear in multiple models if it is similar enough (compression!) • Precision vs recall is generated internally by the database

  13. Control tests: rotation invariance • Match is somewhat sensitive to noise… e.g. higher frequency noise introduced by the sampling of the contours

  14. Control tests: rotation invariance • Tinkering with filtering and other stuff doesn’t help much… using the RDF does.

  15. Control tests: Scale invariance

  16. Circular parameterization andModel match option 1 20 classes shown57 total classes622 models

  17. Circular parameterization andModel match option 1 20 classes shown57 total classes622 models

  18. Circular parameterization andModel match option 1 20 classes shown57 total classes622 models

  19. Radial distance parameterization andModel match option 1 15 classes shown57 total classes622 models 1 silhouette used in query

  20. Radial distance parameterization andModel match option 1 15 classes shown57 total classes622 models 2 silhouettes used in query

  21. Radial distance parameterization andModel match option 1 15 classes shown57 total classes622 models 8 silhouettes used in query

  22. Radial distance parameterization andModel match option 2 1 silhouette used in query 15 classes shown57 total classes622 models

  23. Radial distance parameterization andModel match option 2 4 silhouettes used in query 15 classes shown57 total classes622 models

  24. Radial distance parameterization andModel match option 2 4 silhouettes used in query 15 classes shown57 total classes622 models

  25. Radial distance parameterization andModel match option 2 8 silhouettes used in query15 classes shown57 total classes622 models

  26. Radial distance parameterization andModel match option 2 8 silhouettes used in query, 57 classes shown57 total classes622 models

More Related