1 / 40

by Rene Vidal

Multiple View Geometry Unified. Yi Ma (UIUC), Kun Huang (UIUC) and Jana Kosecka (GMU). by Rene Vidal. Electrical Engineering & Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu. FORMULATION: camera model and multiple images.

jacquelinew
Download Presentation

by Rene Vidal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multiple View Geometry Unified Yi Ma (UIUC), Kun Huang (UIUC) and Jana Kosecka (GMU) by Rene Vidal Electrical Engineering & Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu

  2. FORMULATION: camera model and multiple images ALGEBRA: multilinear constraints v.s. rank deficiency condition GEOMETRY: geometric interpretation of rank deficiency condition ALGORITHM: matching, transfer, motion and structure recovery GENERALIZATION: line features, 3-D curves and surfaces

  3. FORMULATION - Fundamental Geometric Problem Input: Corresponding images (of point or line) in multiple images. Output: Camera motion, camera calibration, object 3D structure.

  4. FORMULATION – Literature Review Multiple view geometry theory • Two views: Longuet-Higgins’81, Huang & Faugeras’89, … • Three views: Spetsakis & Aloimonos’90, Shashua’94, Hartley’94, … • Four views: Triggs’95, Shashua’00, … • Multiple views: Heyden & Astrom’97’98, Ma et. al.’99, … Multiple view geometry algorithms • Euclidean: Maybank’93, Weng, Ahuja & Huang’93, … • Affine: Quan & Kanade’96, … • Projective: Triggs’96, … • Orthographic: Tomasi & Kanade’92, … Recent books on multiple view geometry 1. Multiple view geometry in computer vision, Hartley & Zisserman’00. 2. Geometry of multiple images, Faugeras & Luong’01.

  5. surface curve line point practice projective algorithm affine theory Euclidean 2 views algebra 3 views geometry 4 views optimization m views FORMULATION – An Anatomy of Cases (State of the Art)

  6. surface curve line point practice projective algorithm affine theory Euclidean rank deficiency 2 views algebra 3 views geometry 4 views optimization m views FORMULATION – A Need for Unification

  7. FORMULATION – Pinhole Camera Model Homogeneous coordinates of a 3-D point Homogeneous coordinates of its 2-D image Projection of a 3-D point to an image plane

  8. FORMULATION – Hat Operator

  9. FORMULATION – Multiple View Structure From Motion Given corresponding images of points: recover everything else from equations: “incidental condition” . . .

  10. FORMULATION: camera model and multiple images ALGEBRA: multilinear constraints v.s. rank deficiency condition GEOMETRY: geometric interpretation of rank deficiency condition ALGORITHM: matching, transfer, motion and structure recovery GENERALIZATION: line features, 3-D curves and surfaces

  11. ALGEBRA – Multilinear Constraints For images of the same 3-D point : is rank deficient (leading to the conventional approach) Multilinear constraints among 2, 3, 4 views

  12. ALGEBRA – Rank Deficiency of the Multiple View Matrix WLOG, choose camera frame 1 as the reference Multiple View Matrix Theorem [Rank Deficiency Condition] (generic) (degenerate) Let then and are linearly dependent.

  13. ALGEBRA – M Matrix Implies Bilinear Constraints Fact: Given non-zero vectors Hence, we have These constraints are only necessary but NOT sufficient!

  14. ALGEBRA – M Matrix Implies Trilinear Constraints Fact: Given non-zero vectors Hence, we have • These constraints are only necessary but NOT sufficient! • However, there is NO further relationship among any 4 views. • Quadrilinear constraints hence do not exist!

  15. FORMULATION: camera model and multiple images ALGEBRA: multilinear constraints v.s. rank deficiency condition GEOMETRY:geometric interpretation of rank deficiency condition ALGORITHM: matching, transfer, motion and structure recovery GENERALIZATION: line features, 3-D curves and surfaces

  16. GEOMETRY – Uniqueness of Pre-image by Bilinear Constraints “Bilinear means pair-wise coplanar”: except in a rare coplanar case: Rectilinear motion Trifocal plane

  17. GEOMETRY – Uniqueness of Pre-image by Trilinear Constraints “Trilinear means triple-wise incidental”: except in a rare collinear case:

  18. GEOMETRY – Uniqueness of Pre-image by M Matrix Theorem [Uniqueness of Pre-image] Given vectors with respect to camera frames, they correspond to a unique point in the 3-D space if the rank of the matrix is of rank 1. If the rank is 0, the point is determined up to a line on which all the camera centers must lie. “incidental condition” . . .

  19. GEOMETRY – Geometric Interpretation of M Matrix is the “depth” of the point relative to the camera center. Points that give the same matrix are on a sphere of radius

  20. FORMULATION: camera model and multiple images ALGEBRA: multilinear constraints v.s. rank deficiency condition GEOMETRY: geometric interpretation of rank deficiency condition ALGORITHM:matching, transfer, motion and structure recovery GENERALIZATION: line features, 3-D curves and surfaces

  21. ALGORITHM 1 – Multiple View Matching Test Given the projection matrix associated to camera frames. Then for vectors

  22. ALGORITHM 2 – Motion and Structure from Multiple Views Given images of points:

  23. ALGORITHM 2 – SVD Based Four Step Algorithm

  24. ALGORITHM 2 – Simulation Results Motion XX-YY, 1000 trials and T/R ratio 1.5

  25. ALGORITHM 2 – Simulation Results Motion XX-YY-ZZ, 1000 trials and T/R ratio 1.5

  26. ALGORITHM 3 – Mapping Images to a New View Given the projection matrix associated to camera frames. Then for given vectors So given images, rank deficiency adds a linear constraint on the image. Computing the kernel of gives the new image.

  27. FORMULATION: camera model and multiple images ALGEBRA: multilinear constraints v.s. rank deficiency condition GEOMETRY: geometric interpretation of rank deficiency condition ALGORITHM: matching, transfer, motion and structure recovery GENERALIZATION:line features, 3-D curves and surfaces

  28. GENERALIZATION – Line Features Homogeneous representation of a 3-D line Homogeneous representation of its 2-D image Projection of a 3-D line to an image plane

  29. GENERALIZATION – Multiple View Matrix: Line v.s. Point Point Features Line Features

  30. GENERALIZATION – Point/Point Duality Point/point duality between a camera center and a 3-D point: Theorem [Point/Point Duality] From matrix only, if a camera center is moving on a straight line, a fixed 3-D point is determined up to a circle; if a camera center is fixed but a point is moving on a line, the line is determined up to a circle.

  31. GENERALIZATION – Line Features (v.s. Point Features)

  32. GENERALIZATION – SFM from Line Features Given images of lines:

  33. GENERALIZATION – Planar Features Homogeneous representation of a 3-D plane Projection of a planar point to the image Projection of a planar line to the image

  34. GENERALIZATION – Multiple View Matrix: Coplanar Features Given that a point and line features lie on a plane in 3-D space: Besides multilinear constraints, it simultaneously gives homography:

  35. GENERALIZATION – Coplanar Point/Line Duality On the plane any two points determine a line and any two lines determine a point. Theorem [Point/Line Duality] For planar features, points and lines are hence equivalent!

  36. GENERALIZATION – SFM from Coplanar Features On the plane a set of points is equivalent to a set of lines, vice versa. • One can use either planar or to solve SFM as in the generic • point and line case. Algorithms need only minor changes. • Rank deficiency of planar or exploits multilinear constraints • and homography constraints simultaneously.

  37. GENERALIZATION – 3-D Curves & Surfaces Differentiating the matrix of a point (moving) along a curve: gives rise to rank deficiency condition for curve. intensity level sets region boundaries . . .

  38. GENERALIZATION – From Tangent to Point-wise Correspondence gives rise to a set of ordinary differential equations: • The rank deficiency condition for relates points and tangent • lines of a curve. • Solving these equations establishes point-wise correspondence for • image curves and in fact eventually for surface as well. • gives constraints on curvature and normals of image curves.

  39. CONCLUSIONS AND ON-GOING WORK • Rank deficiency condition simplifies and unifies existing algebraic • results in multilinear constraints (no tensor and algebraic geometry). • Rank deficiency condition exhibits clear geometric interpretation. • Rank deficiency condition unifies the study of point, line, curve and • even surface in 3-D. • Rank deficiency condition naturally reveals point/point and • point/line duality. • Rank deficiency condition gives rise to uniform linear algorithms for • feature matching, motion recovery and new view synthesis. • The results no longer discriminate two, three, four or multiple views, • nor Euclidean, affine or projective camera models. • Consistent, optimal and robust reconstruction of motion/structure. • Numerical algorithms for curve, surface reconstruction from m views. • Multiple views of multiple rigid body motions.

  40. Multiple View Geometry Unified Kun Huang, Rene Vidal and Jana Kosecka by Yi Ma CSL Technical Report, UILU-ENG #01-2208 (DC-200), 05/08/01 CSL Technical Report, UILU-ENG #01-2209 (DC-201), 05/08/01

More Related