Research On 3D Model Retrieval

1. Research On 3D Model Retrieval Pu Jiantao. pjt@cis.pku.edu.cn October, 2003. Department of Intelligent Science, Peking University 版权所有，未经允许，禁止使用

2. Content • Definition • Motivation • Difficulty • Related Works • Our Research Objective • Outlines of Our Methods • Organization • Summary

3. Definition: what’s 3D Model Retrieval? • Its goal is to find out the models with the most desired geometric shape from 3D model database. 3D Model Database Query Feature? Result

4. Definition: how to do query? • 3D model can be describe by: • Text about 3d model’s properties, such as name, color, texture, material, function, etc. • Shape: content-base.

5. Definition: how to do query? • Example 1: How do you differentiate following cars precisely by the way of text?

6. Definition: how to do query? • Example 2: Following is a model with arbitrary shape, how do you describe it precisely?

7. Definition: how to do query? • Example 3: The top row is a base-model, the bottom row is a serials of models arrayed in similarity order. How do you do this precisely? High Similarity Low Similarity

8. Definition: conclusion? • Key-words based query is not a good way. • Geometric shape is the most powerful way in 3D model description, as it is the most intuitive way that human apperceive 3D models. So shape matching is the best way to do 3D model retrieval. However, it is a very difficult problem.

9. Definition: conclusion? Framework of 3D Model Retrieval System:

10. Definition: conclusion? Main Research Targets: • Research on Feature Definition and Extraction for 3D Models; • Research on Similarity Measure; • Design and Implementation of Query Interface; • Rapid Search Methods;

11. Definition: conclusion? • Shape matching is to extract and compare the key features between 3D models.

12. 2D Image Simple contour representation Features occlusion, shadow, noise Camera dependent …… Definition: the difference between 3D Model Retrieval & 2D Image Retrieval? • 3D Model • Complex surface representation • No features occlusion, shadows, noise • Camera independent • …….

13. Motivation • 3D models can be easily obtained: • 3D modeler • 3D Scanner • Rebuilt by several images • Commercial 3D model library

14. Motivation • 3D models are widely used in many fields: • Industry • Games • Art • Virtual Reality • E-business • Education • Architecture

15. Motivation • 3D technologies are developing rapidly. • The number of 3D models is increasing at a surprising rate, especially under the stimulation of Internet.

16. Difficulties • It is hard to define and extract the features of 3D models; • It is hard to bridge the gap between shape and Semantics; • It is hard to measure the similarity between models; • It is hard to do shape matching between models that may be in arbitrary pose.

17. Feature? Feature? Feature? Difficulties • It is hard to define and extract the features of 3D models;

18. Difficulties • It is hard to bridge the gap between shape and Semantics; Similar Hammer Toy: Drum How do you know it is a hammer or drum???

19. Difficulties • It is hard to measure the similarity between models; Similar Hammer Toy: Drum To which extend above models are similar???

20. Difficulties • It is hard to do shape matching between models that may be in arbitrary pose.

21. Difficulties • Good features should have following characteristics: • quick to compute • concise to store • easy to index • invariant under similarity transforms • insensitive to noise and small extra features • independent of 3D object representation • robust to arbitrary topological degeneracies • discriminating of shape similarities and differences

22. Related Works • All methods for shape matching can be roughly classified as: (1) Feature vector-based; (2) Statistics-based; (3) Topology-based;

23. Related Works: Feature vector-based Method • Features include: Area, circularity, eccentricity, compactness, major axis orientation, Euler number, concavity, etc. Database Feature Vector Vector Comparison Similar Models Features

24. Related Works: Feature vector-based Method • The idea is simple, but the implementation is tedious. In addition, different models have different key features.

25. Related Works: Feature vector-based Method • M. Hebert, K. Ikeuchi and H. Delingette. A Spherical Representation for Recognition of Free-form Surfaces. IEEE Trans. PAMI, Vol.17, pp.681-690, 1995. • R. Sonthi, G. Kunjur and R. Gadh. Shape Feature Determination using the Curvature Region Representation. Proc. Symp. Solid Modeling, pp.285-296, 1997. • G. Dudek and J.K. Tsotsos. Shape Representation and Recognition from Multiscale Curvature. Computer Vision and Image Understanding, Vol.68, pp.170-189, 1997. Discrete Geodeic Problem. SIAM J. Computing, Vol.16, pp.647-667, 1987.

26. Related Works: Statistics-based Method • Shape Distribution Robert O., Thomas F., Bernard C., and David D., Shape Distribution. ACM Transactions on Graphics, 21(4), pp. 807-832, October 2002. The key idea is to represent the signature of an object as a shape distribution sampled from a shape function measuring global geometric.

27. Related Works: Statistics-based Method • (cont.)Shape Distribution It is simpler than traditional shape matching methods that require pose registration, feature correspondence, or model fitting.

28. Related Works: Statistics-based Method • (cont.) Three steps are involved: (1) Selecting a shape function: A3：Measures the angle between three random points on the surface of a 3D model. D1：Measures the distance between a fixed point and one random point on the surface. Generally, the centroid of the boundary of the model is used as the fixed point. D2：Measures the distance between two random points on the surface. D3：Measures the distance between two random points on the surface. D4：Measures the cube root of the volume of the tetrahedron between four random points on the surface.

29. Related Works: Statistics-based Method • (cont.) (1) Selecting a shape function: (cont.) Example D2 shape distributions.

30. Related Works: Statistics-based Method • (cont.) (2) Sampling method The time to sample a shape distribution is linearly proportional to the number of samples. Sampling a random point in a triangle.

31. Related Works: Statistics-based Method • (cont.) (2) Comparing Shape Distribution

32. Related Works: Statistics-based Method • (cont.) (2) Comparing Shape Distribution(cont.) (1) bottleneck distance; (2) Hausdorff distance; (3) Turing function distance; (4) Fréchet Distance; (5) nonlinear elastic matching distance; (6) reflection distance; (7) area of symmetric difference; (8) transport distance; (9) Earth Mover’s distance; (10) discrete distance; As for detailed definitions, you can refer to:Veltkamp, R. C., Shape matching: Similarity measures and algorithms. In Shape Modelling International (Genova), 188–197, 2001.

33. Related Works: Statistics-based Method • Ryutarou Ohbuchi, Shape-Similarity Search of Three-Dimensional Models Using Parameterized Statistics,Accepted for publication in the proceedings of the Pacific Graphics 2002, Beijing, China, October 9-11, 2002.

34. Related Works: Statistics-based Method • (Cont.)

35. Related Works: Statistics-based Method • (Cont.) Experiment Results: Euclidean distance. Elastic matching.

36. Related Works: Statistics-based Method • An Example: • Sphere is evenly sampled • For each sphere sample min. distance to object calculated Signature Comparison: L1,L2,L∞

37. Related Works: Topology-based Method • Topology is a compact representation of 3D models: (1) Intuitive; (2) Flexibility: global & local; (3) Transform Invariance;

38. Related Works: Topology-based Method • H. Sundar,D. Silver, . Gagvani, S. Dickinson, Skeleton Based Shape Matching and Retrieval, Shape Modeling International 2003, Seoul , Korea, May 12 - 15, 2003.

39. Related Works: Topology-based Method • (Cont.) Skeleton Creation Distance Based Directed Acyclic Graph

40. Related Works: Topology-based Method • (cont.)Shape Graph Matching Two factors determine whether two nodes of the trees get matched: the first is a measure of the topological similarity of the subtrees rooted at the nodes, while the second is a measure of the local shape information at that node.

41. Related Works: Topology-based Method • (Cont.) But this method has a high computational cost and is sensitive to noise and small undulations.

42. Related Works: Topology-based Method • Hilaga, M., Shinaagagawa, Y., Kohmura, T., and Kunii, T. L., Topology Matching for Fully Automatic Similarity Estimation of 3D Shapes. In Proceedings of SIGGRAPH 2001. Computer Graphics Proceedings, Annual Conference Series, 203–212, 2001. • Multi-resolution Reeb graph is used as a search key that represents the features of a 3D shape. Torus and its Reeb graph using a height function

43. Related Works: Topology-based Method • Generally, a simple reeb graph is created by using height function • μ - height of the point V: μ(V(x,y,z))=z

44. Related Works: Topology-based Method • The basic idea of the MRG is to develop a series of Reeb graphs for an object at various levels of detail.

45. Related Works: Topology-based Method • The building process of MRG: • Subdivision • Interpolate the position of two relevant vertices in the same proportion as their value of µn(v)

46. Related Works: Topology-based Method • The building process of MRG: • Calculate T-sets • Connect R-nodes

47. Related Works: Topology-based Method • The building process of MRG: • Construct MRG • fine-to-coarse (reverse)

48. Related Works: Topology-based Method • The algorithm does not distinguish between left and right. • Remaining Issues: The structure of an MRG is sensitive to the placement of the region boundaries.

49. Related Works: Topology-based Method • Actually, Height function is not appropriate • not invariant to transformations. • Instead, a geodesic distance is used: • Not invariant to scale: • Normalize [0,1]:

50. Related Works: Topology-based Method • http://shape.cs.princeton.edu