1 / 46

From Digital Photos to Textured 3D Models on Internet

From Digital Photos to Textured 3D Models on Internet. Gang Xu Ritsumeikan University & 3D Media Co., Ltd. On-site Demo 3DM Modeler 3DM Stitcher 3DM Calibrator 3DM Viewer. Applications. Cultural heritage modeling Car & ship modeling 3D Survey Plant Modeling Game Virtual Reality

alexia
Download Presentation

From Digital Photos to Textured 3D Models on Internet

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. From Digital Photos to Textured 3D Models on Internet Gang Xu Ritsumeikan University & 3D Media Co., Ltd.

  2. On-site Demo3DM Modeler3DM Stitcher3DM Calibrator3DM Viewer

  3. Applications • Cultural heritage modeling • Car & ship modeling • 3D Survey • Plant Modeling • Game • Virtual Reality • Web design

  4. Outline of This Tutorial • How can 3D be recovered from 2D? • Mathematical Preparations: Solving Linear Equations and Non-linear Optimization • Camera Model and Projection • Epipolar Geometry Between Images • Linear Algorithm for 2-View SFM • Bundle Adjustment (non-linear optimization)

  5. General Strategy • Step 1: Find a linear or closed-form solution • Step 2: Non-linear optimization • A linear or closed-form solution via many intermediate representations usually includes large errors. But this solution can serve as the initial guess in the non-linear optimization.

  6. Part I Mathematical Preparation

  7. Solving Linear Equations (1)

  8. Solving Linear Equations (2) • When n=2 • When n>2, the inverse of A does not exist.

  9. Solving Linear Equations (3) Best compromise Define Minimize A+ : pseudo-inverse matrix of A

  10. Solving Linear Homogeneous Equations • When c=0, we cannot use x Define Minimize Under ||x||=1 Solution: Eigen vector associated with the smallest eigen value of

  11. An Example (1) • Fitting a line to a 2D point set • Plane equation • Minimize

  12. An Example (2) x0 e1 a e2

  13. Non-Linear Optimization (1) • Minimize non-linear function

  14. Non-Linear Optimization (2) • High-dimensional space • No closed-form solution • Multiple local minima • Iterative procedure, starting from an initial guess, assuming that f is differentiable everywhere.

  15. An Example: Bundle Adjustment Bundle Adjustment: minimizing difference between observations and back-projections

  16. 3 Common Used Algorithms • Gradient Method (1st-order derivative) • Newton Method (2nd-orde derivative) • Levenberg-Marquartd Method = Gradient+Newton, used in 3DM Modeler, 3DM Stitcher and 3DM Calibrator

  17. Part II Camera and Projection

  18. Pinhole Camera • Pinhole Camera lens center Image

  19. Projection Formula y Y y (X,Y,Z) Focal length Z Principal point focus Mirror image Real image

  20. Projection Matrix Homogeneous Coordinates

  21. u Principal point x (u0,v0) v y Image Coordinate Systems Intrinsic parameters Principal point approximately at the image center A: Intrinsic matrix

  22. 3D Coordinate Transform Z’ X’ Y’ Object coordinate system Rotation R Translation t Z X Y Camera coordinate system

  23. From Object Coordinates to Pixel Coordinates Z’ X’ u Y’ v F

  24. What Is Unknown? • Intrinsic Parameters: focal length f • Extrinsic Parameters: Rotation matrix R, translation vector t

  25. Rotation Representations r/||r|| ||r|| • 3×3 Rotation MatrixR • Satisfying • There are only 3 degrees of freedom. • A rotation can be represented by a 3-vector r, with ||r|| representing the rotation amount, and r/||r|| representing the rotation axis. • Others: Euler Angles, Roll-Pitch-Yaw, Quarternion

  26. Rodrigue’s Formula

  27. Part III Epipolar Geometry, Essential Matrix, Fundamental Matrix and Homography Matrix

  28. Epipolar Geometry Between Two Images Epipolar lines Epipolar lines Epipolar plane epipoles F F’

  29. Epipolar Equation F,F’,x,x’ are coplanar. x x’ F F’ t

  30. Essential Matrix E: Essential Matrix

  31. Fundamental Matrix F: Fundamental matrix

  32. Degenerate Cases Lead to Homography • Case 1: t=0 (no translation, only rotation. Panorama- 3DM Stitcher) • Case 2: planar scene (3DM Calibrator) • In both cases: the image can be transformed into another by a homography.

  33. Determining E, F and H Matricesfrom Point Matches • E, F and H can only be determined up to scale. • F (and E) can be linearly determined from 8 or more point matches. • H can be linearly determined from 4 or more point matches.

  34. Part VI A Linear Algorithm for 2-View SFM

  35. Determining Translation from Essential Matrix • Assuming that the intrinsic matrix A is available • First compute Essential matrix • Determining t (up to a scale) from • There are two solutions t and -t

  36. Determining Rotation R • We can consider the problem as determining R from 3 pairs of vectors before and after the rotation

  37. Determining Rotation from n Pairs of Vectors R • Given (pi,pi’), i=1,…,n (n>=2), • Define and minimize • There is a linear solution using SVD. … …

  38. F F’ R,t Determining 3D coordinates • Given R, t, x and x’, determine unknown scales s and s’ such that the two lines “meet” in the 3D space.

  39. F F’ R,t In Front of Both Cameras • s>0, s’>0 in front of both cameras • s>0, s’<0 • s<0, s’>0 • s<0, s’<0

  40. Part V Bundle Adjustment

  41. Coordinate System • All space points and camera positions & camera poses are defined in a common coordinate system.

  42. Match Matrix • Match Matrix w(i,j)=1, if i-th point appears in j-th image; otherwise w(i,j)=0 j 1 2 3 … i 0 1 1 1 1 2 . . 1 1 0

  43. Cost Function

  44. Parameters to Determine • The parameters to determine are • (Xi,Yi,Zi), i=1,…,n. Totally 3n • (tj,rj,fj), j=1,…,m. Totally 7m • Altogether 3n+7m

  45. Initial Guess • LM method needs good initial guesses. • They can be obtained using the previous linear algorithm. • Needs to transform all data into the common coordinate system.

  46. References • Olivier Faugeras: 3D Computer Vision: A Geometry Viewpoint, MIT Press, 1993 • Gang Xu and Zhengyou Zhang: Epipolar Geometry in Stereo, Motion and Object Recognition, Kluwer, 1996 • Richard Hartley and Andrew Zisserman: Multiple View Geometry, Cambridge Univ Press, 2000

More Related