Lecture 8 Feature Matching and Stereo Vision

1. Lecture 8Feature Matching and Stereo Vision Slides by: Clark F. Olson Jean Ponce Linda G. Shapiro

2. Shape from single images • There are many image cues from which we can determine the shape of objects in the scene. • What can we determine from a single image?

3. Shape from X • Methods for extracting shape in computer vision include: • Silhouette • Shading • Texture • Focus • Structured lighting • Motion • Stereo

4. Stereo Vision • Stereo vision is the ability to infer the 3D structure of a scene using two (or more) images from different viewpoints. • Two fundamental problems: • Correspondence problem Which points are the projection of the same scene element? • Reconstruction problem Given a correspondence (and the camera geometry) what is the 3D location of the observed object? • For precise measurements, calibration is necessary.

5. Epipolar Geometry • Epipolar Plane • Baseline • Epipolar Lines • Epipoles

6. Epipolar Constraint • Potential matches for p have to lie on the correspondingepipolarline l’. • Potential matches for p’ have to lie on the corresponding epipolarline l.

7. Stereo Reconstruction If we know the corresponding points in the two images, we can determine where the point is relative to the cameras. This assumes that we know precisely where the cameras are relative to each other!

8. Binocular Fusion Usually, we don’t know in advance which points correspond to each other. If we get the correspondence wrong, then we will compute the wrong point. Finding the correspondences is an important problem.

9. Reconstruction If our correspondences are precisely correct, then the rays from image points through the optical centers will intersect. In practice, this is rare. Solution: find the point that minimizes the distance to the two rays.

10. Rectification A useful trick: If we reproject the images carefully (onto a common plane parallel to the baseline) then the epipolar lines are horizontal. We will assume that this has been done.

11. Rectification Example Input left image: Not rectified Input right image: Not rectified After rectification The red lines show corresponding scanlines.

12. Stereo Disparity 3D point left image right image Disparity: the difference in image location of the same 3D point when projected under perspective to two different cameras. d = xleft - xright

13. Correspondence How do we determine correspondences between (rectified) images? Usually by comparing small image patches. ? Right image Left image

14. Correspondence Far away points move little and look similar. Nearby points move a lot and look dissimilar. (Assumes rectified images.)

15. Dense Stereo Dense stereo information can be computed by asking for each pixel in the left image: what is the corresponding pixel in the right image? Left image Color-coded range map Right image Range map after pruning likely mismatches