1 / 35

Sketch-based 3D shape retrieval

Mathias Eitz , Kristian Hildebrand, Tamy Boubekeur and Marc Alexa. Sketch-based 3D shape retrieval. Goal: sketch-based shape retrieval. input sketch. query result. query. retrieve. 3D model database. Outline. Background Sketches as input Overview Framework Results.

tarala
Download Presentation

Sketch-based 3D shape retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mathias Eitz, Kristian Hildebrand, TamyBoubekeur and Marc Alexa Sketch-based 3D shape retrieval

  2. Goal: sketch-based shape retrieval input sketch query result query retrieve 3D model database

  3. Outline • Background • Sketches as input • Overview • Framework • Results

  4. Why sketches as input? • 3 common strategies for input quick, simple, semantics keywords no/incorrect tags rather simple, independent of external data model (3D) requires drawing skills rich input leads to good results sketch (2D) example often not available

  5. Why sketches as input? • Shape parts index into human memory [Hoffman’97] • 80-90% of lines explained by known definitions [Cole’08] human sketch computer ≈

  6. Outline • Background • Overview • Previous work • Comparison with our approach • Framework • Results

  7. Overview • Current retrieval systems rely on common scheme: ≈ s s , , 0.21 0.23 0.21 0.13 0.15 0.13 0.75 0.78 0.75 0.31 0.29 0.31 0.41 0.40 0.41 … … … 0.17 0.15 0.17

  8. Previous work: global features [Löffler, 2000] global analysis global descriptor [Funkhouser’03] [Pu’05] [Hou’07] [Shin’07] [Napoleon’09] 0.21 0.13 0.75 0.31 0.41 … 0.17

  9. Our approach: local features • Independent local features allow for: • translation invariance • partial matching • standard search data structures ... ... Bag-of-features [Sivic’03]

  10. Outline • Background • Overview • Framework • Offline indexing • Learning visual vocabulary • Online search • Results

  11. Offline indexing generate render ... ... ... ...

  12. Offline indexing Visual vocabulary extract ... render quantize ... 2 0 0 add for all 50 views separately 7 0 0

  13. Offline indexing: view generation • Uniformly sample bounding sphere: 50 samples ... ...

  14. Offline indexing: NPR lines • Occluding contours • Suggestive contours [DeCarlo’03]

  15. Offline indexing: sampling & features • Sampling: 500 random samples on lines • Representation: should be concise & robust • local image statistics ...

  16. Offline indexing: features • No directionality information in gradients • Binned distribution invariant to small deformations (1) Extract local region (2) estimate orientations 4x4 spatial, 8 radial bins (3) distribution of orientations

  17. Offline indexing: visual vocabulary • 20k images (sampled from 50 views each of 2k models) • 500 local features each • Training set size: 10 million local features . . . training set 20k random images

  18. Offline indexing: visual vocabulary training set : k-means … k=500 Cluster centers form “visual vocabulary” ... , , , , 499 2 1 0 id:

  19. Offline indexing: quantization Feature to be quantized • Quantization allows for • More compact representation • Grouping of perceptually similar features visual word 499 represented by ... , , , , 499 2 1 0 id:

  20. Offline indexing: representation ... ... ... ... ... ... ... , , , , 1 0 1 1 1 1 0 0 0 0 0 0 0 2 1 2 0 1 1 0 2 0 1 0 histogram of “visual words” 499 2 1 0 id:

  21. Offline indexing: representation = ... . . . 1 0 2 2

  22. Online search Visual vocabulary extract ... render quantize ... 2 0 0 query add for all 50 views separately 7 0 0

  23. Online Search 500 feature vectors 2 sample 500 locations 0 0 7 quantize 3 0 0 3

  24. Online search • Images as (sparse) histograms of visual words # words word id

  25. Online search • Similarity as angle in high-dimensional space • Vectors sparse: use inverted index 2 2 0 0 s 0 3 6 7 , , 3 0 0 0 0 0 3 2

  26. Outline • Background • Overview • Framework • Results • Images • Discussion

  27. Results • Based on Princeton shape db (~2k models) • ~10ms for a search

  28. Results

  29. Results

  30. Failure cases • NPR methods require high resolution meshes • Sketches from “real users” can be quite abstract

  31. Future work • View generation • canonical, “salient” views – which provides best retrieval? • Feature representation • multi-scale, rotation-invariance? • Larger datasets than the PSB models • Method fast enough to handle millions of models • Will it remain effective?

  32. Thanks • Acknowledgements • Princeton shape benchmark [Shilane’04] • RTSC tool by Doug DeCarlo, SzymonRusinkiewicz • Cited authors for images from their papers

More Related