1 / 37

Recognition and Matching based on local invariant features

Recognition and Matching based on local invariant features. Cordelia Schmid INRIA, Grenoble. David Lowe Univ. of British Columbia. ( ). Introduction. Local invariant photometric descriptors. local descriptor. Local : robust to occlusion/clutter + no segmentation

althea
Download Presentation

Recognition and Matching based on local invariant features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Recognition and Matching based on local invariant features Cordelia Schmid INRIA, Grenoble David Lowe Univ. of British Columbia

  2. ( ) Introduction Local invariant photometric descriptors local descriptor Local : robust to occlusion/clutter + no segmentation Photometric : distinctive Invariant : to image transformations + illumination changes

  3. History - Matching Matching based on line segments • Not very discriminant • Solution : matching with interest points & correlation [ A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry, Z. Zhang, R. Deriche, O. Faugeras and Q. Luong, Artificial Intelligence 1995 ]

  4. Approach • Extraction of interest points with the Harris detector • Comparison of points with cross-correlation • Verification with the fundamental matrix

  5. Harris detector Interest points extracted with Harris (~ 500 points)

  6. Cross-correlation matching Initial matches (188 pairs)

  7. Global constraints Robust estimation of the fundamental matrix 89 outliers 99 inliers

  8. Summary of the approach • Very good results in the presence of occlusion and clutter • local information • discriminant greyvalue information • robust estimation of the global relation between images • for limited view point changes • Solution for more general view point changes • wide baseline matching (different viewpoint, scale and rotation) • local invariant descriptors based on greyvalue information

  9. History - Recognition Color histogram [Swain 91] Each pixel is described by a color vector Distribution of color vectors is described by a histogram => not robust to occlusion, not invariant, not distinctive

  10. History - Recognition Eigenimages [Turk 91] • Each face vector is represented in the eigenimage space • eigenvectors with the highest eigenvalues = eigenimages • The new image is projected into the eigenimage space • determine the closest face . . . . • not robust to occlusion, requires segmentation, not invariant, discriminant

  11. History - Recognition Geometric invariants [Rothwell 92] • Function with a value independent of the transformation • Invariant for image rotation : distance of two points • Invariant for planar homography : cross-ratio where => local and invariant, not discriminant, requires sub-pixel extraction of primitives

  12. History - Recognition Problems : occlusion, clutter, image transformations, distinctiveness • Solution : recognition with local photometric invariants [ Local greyvalue invariants for image retrieval, C. Schmid and R. Mohr, PAMI 1997 ]

  13. ( ) Approach 1) Extraction of interest points (characteristic locations) 2) Computation of local descriptors 3) Determining correspondences 4) Selection of similar images local descriptor

  14. Interest points Geometric features repeatable under transformations 2D characteristics of the signal high informational content Comparison of different detectors [Schmid98] Harris detector

  15. Harris detector Based on the idea of auto-correlation Important difference in all directions => interest point

  16. Harris detector Auto-correlation function for a point and a shift Discret shifts can be avoided with the auto-correlation matrix

  17. Harris detector Auto-correlation matrix

  18. Harris detection • Auto-correlation matrix • captures the structure of the local neighborhood • measure based on eigenvalues of this matrix • 2 strong eigenvalues => interest point • 1 strong eigenvalue => contour • 0 eigenvalue => uniform region • Interest point detection • threshold on the eigenvalues • local maximum for localization

  19. ( ) Local descriptors local descriptor Descriptors characterize the local neighborhood of a point

  20. Local descriptors Greyvalue derivatives

  21. Local descriptors Invariance to image rotation : differential invariants [Koen87]

  22. Local descriptors Robustness to illumination changes In case of an affine transformation

  23. Local descriptors Robustness to illumination changes In case of an affine transformation or normalization of the image patch with mean and variance

  24. ( ) ( ) Determining correspondences ? = Vector comparison using the Mahalanobis distance

  25. Selection of similar images • In a large database • voting algorithm • additional constraints • Rapid acces with an indexing mechanism

  26. ( ) vector of local characteristics Voting algorithm

  27. 1 1 0 1 1 2 2 1 1 I is the corresponding model image 1 Voting algorithm } }

  28. Additional constraints • Semi-local constraints • neighboring points should match • angles, length ratios should be similar • Global constraints • robust estimation of the image transformation (homogaphy, epipolar geometry) 1 1 2 2 3 3

  29. Results database with ~1000 images

  30. Results

  31. Results

  32. Summary of the approach • Very good results in the presence of occlusion and clutter • local information • discriminant greyvalue information • invariance to image rotation and illumination • Not invariance to scale and affine changes • Solution for more general view point changes • local invariant descriptors to scale and rotation • extraction of invariant points and regions

  33. Approach for Matching and Recognition • Detection of interest points/regions • Harris detector (extension to scale and affine invariance) • Blob detector based on Laplacian • Computation of descriptors for each point • Similarity of descriptors • Semi-local constraints • Global verification

  34. Approach for Matching and Recognition • Detection of interest points/regions • Computation of descriptors for each point • greyvalue patch, diff. invariants, steerable filter, SIFT descriptor • Similarity of descriptors • correlation, Mahalanobis distance, Euclidean distance • Semi-local constraints • Global verification

  35. Approach for Matching and Recognition • Detection of interest points/regions • Computation of descriptors for each point • Similarity of descriptors • Semi-local constraints • geometrical or statistical relations between neighborhood points • Global verification • robust estimation of geometry between images

  36. Overview 8:30-8:45 Scale invariant interest points 8:45-9:00 SIFT descriptors 9:00-9:25 Affine invariance of interest points + applications 9:25-9:45 Evaluation of interest points + descriptors 9:45-10:15 Break

  37. Overview 10:15-11:15 Object recognition system, demo, applications 11:15-11:45 Recognition of textures and object classes 11:45-12:00 Future directions + discussion

More Related