1 / 73

Motion Illusion: Rotating Snakes and Local Features

This tutorial explores the main components of local feature detection, extraction of feature descriptors, and matching of feature vectors for correspondence using the concepts of motion illusion and rotating snakes.

kellid
Download Presentation

Motion Illusion: Rotating Snakes and Local Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motion illusion, rotating snakes

  2. Local features: main components • Detection:Find a set of distinctive key points. • Description: Extract feature descriptor around each interest point as vector. • Matching: Compute distance between feature vectors to find correspondence. K. Grauman, B. Leibe

  3. Review: Harris corner detector • Approximate distinctiveness by local auto-correlation. • Approximate local auto-correlation by second moment matrix M. • Distinctiveness (or cornerness) relates to the eigenvalues of M. • Instead of computing eigenvaluesdirectly, we can use determinant and trace of M. • E(u, v) • (max)-1/2 • (min)-1/2 James Hays

  4. Trace / determinant and eigenvalues • Given n x n matrix A with eigenvalues λ1…n • R = λ1λ2 – α(λ1+λ2)2 = det(M) – αtrace(M)2

  5. How invariant are harris corners?

  6. Affine intensity change • Intensity scaling:I  aI • R • R • Threshold • x(image coordinate) • x(image coordinate) • I  aI + b • Only derivatives are used => invariance to intensity shiftI I+b • Partially invariant to affine intensity change James Hays

  7. Image translation • Derivatives and window function are shift-invariant. • Corner location is covariant w.r.t. translation James Hays

  8. Image rotation • Second moment ellipse rotates but its shape (i.e., eigenvalues) remains the same. • Corner location is covariant w.r.t. rotation James Hays

  9. Scaling • Corner • All points will be classified as edges • Corner location is not covariant to scaling! James Hays

  10. What is the ‘scale’ of a feature point?

  11. Automatic Scale Selection How to find patch sizes at which f response is equal? What is a good f ? K. Grauman, B. Leibe

  12. Automatic Scale Selection • Function responses for increasing scale (scale signature) Response of some function f K. Grauman, B. Leibe

  13. Automatic Scale Selection • Function responses for increasing scale (scale signature) Response of some function f K. Grauman, B. Leibe

  14. Automatic Scale Selection • Function responses for increasing scale (scale signature) Response of some function f K. Grauman, B. Leibe

  15. Automatic Scale Selection • Function responses for increasing scale (scale signature) Response of some function f K. Grauman, B. Leibe

  16. Automatic Scale Selection • Function responses for increasing scale (scale signature) Response of some function f K. Grauman, B. Leibe

  17. Automatic Scale Selection • Function responses for increasing scale (scale signature) Response of some function f K. Grauman, B. Leibe

  18. What Is A Useful Signature Function f ? 1st Derivative of Gaussian (Laplacian of Gaussian) Earl F. Glynn

  19. What Is A Useful Signature Function f ? • “Blob” detector is common for corners • - Laplacian (2nd derivative) of Gaussian (LoG) Scale space Function response Image blob size K. Grauman, B. Leibe

  20. Find local maxima in position-scale space s5 Find maxima s4 s3  List of(x, y, s) s2 s K. Grauman, B. Leibe

  21. Alternative approach Approximate LoG with Difference-of-Gaussian (DoG). Ruye Wang

  22. Alternative approach Approximate LoG with Difference-of-Gaussian (DoG). 1. Blur image with σ Gaussian kernel 2. Blur image with kσ Gaussian kernel 3. Subtract 2. from 1. = - K. Grauman, B. Leibe

  23. Find local maxima in position-scale space of DoG Find maxima … … ks 2ks - = s  List of(x, y, s) - = ks - = s K. Grauman, B. Leibe Input image

  24. Results: Difference-of-Gaussian • Larger circles = larger scale • Descriptors with maximal scale response K. Grauman, B. Leibe

  25. Maximally Stable Extremal Regions [Matas ‘02] • Based on Watershed segmentation algorithm • Select regions that stay stable over a large parameter range K. Grauman, B. Leibe

  26. Example Results: MSER K. Grauman, B. Leibe

  27. Review: Interest points • Keypoint detection: repeatable and distinctive • Corners, blobs, stable regions • Harris, DoG, MSER

  28. Review: Choosing an interest point detector • Why choose? • Collect more points with more detectors, for more possible matches • What do you want it for? • Precise localization in x-y: Harris • Good localization in scale: Difference of Gaussian • Flexible region shape: MSER • Best choice often application dependent • Harris-/Hessian-Laplace/DoG work well for many natural categories • MSER works well for buildings and printed things • There have been extensive evaluations/comparisons • [Mikolajczyk et al., IJCV’05, PAMI’05] • All detectors/descriptors shown here work well

  29. Comparison of Keypoint Detectors TuytelaarsMikolajczyk 2008

  30. Local Image Descriptors Read Szeliski 4.1 Acknowledgment: Many slides from James Hays, Derek Hoiem and Grauman & Leibe 2008 AAAI Tutorial

  31. Local features: main components • Detection:Find a set of distinctive key points. • Description: Extract feature descriptor around each interest point as vector. • Matching: Compute distance between feature vectors to find correspondence. K. Grauman, B. Leibe

  32. Image Representations: Histograms Global histogram to represent distribution of features • Color, texture, depth, … Local histogram per detected point Space Shuttle Cargo Bay Images from Dave Kauchak

  33. For what things do we compute histograms? • Color • Model local appearance L*a*b* color space HSV color space James Hays

  34. For what things do we compute histograms? • Texture • Local histograms of oriented gradients • SIFT: Scale Invariant Feature Transform • Extremely popular (40k citations) SIFT – Lowe IJCV 2004 James Hays

  35. SIFT • Find Difference of Gaussian scale-space extrema • Post-processing • Position interpolation • Discard low-contrast points • Eliminate points along edges

  36. SIFT • Find Difference of Gaussian scale-space extrema • Post-processing • Position interpolation • Discard low-contrast points • Eliminate points along edges • Orientation estimation

  37. SIFT Orientation Normalization p 2 0 • Compute orientation histogram • Select dominant orientation ϴ • Normalize: rotate to fixed orientation p 2 0 T. Tuytelaars, B. Leibe [Lowe, SIFT, 1999]

  38. SIFT • Find Difference of Gaussian scale-space extrema • Post-processing • Position interpolation • Discard low-contrast points • Eliminate points along edges • Orientation estimation • Descriptor extraction • Motivation: We want some sensitivity to spatial layout, but not too much, so blocks of histograms give us that.

  39. SIFT Descriptor Extraction • Given a keypoint with scale and orientation: • Pick scale-space image which most closely matches estimated scale • Resample image to match orientation OR • Subtract detector orientation from vector to give invariance to general image rotation.

  40. SIFT Orientation Normalization p 2 0 • Compute orientation histogram • Select dominant orientation ϴ • Normalize: rotate to fixed orientation p 2 0 T. Tuytelaars, B. Leibe [Lowe, SIFT, 1999]

  41. SIFT Descriptor Extraction • Given a keypoint with scale and orientation Gradient magnitude and orientation 8 bin ‘histogram’ - add magnitude amounts! Utkarsh Sinha

  42. SIFT Descriptor Extraction • Within each 4x4 window Gradient magnitude and orientation 8 bin ‘histogram’ - add magnitude amounts! Weight magnitude that is added to ‘histogram’ by Gaussian Utkarsh Sinha

  43. SIFT Descriptor Extraction • Extract 8 x 16 values into 128-dim vector • Illumination invariance: • Working in gradient space, so robust to I = I + b • Normalize vector to [0…1] • Robust to I = αI brightness changes • Clamp all vector values > 0.2 to 0.2. • Robust to “non-linear illumination effects” • Image value saturation / specular highlights • Renormalize

  44. Specular highlights move between image pairs!

  45. SIFT-like descriptor in Project 2 • SIFT is hand designed based on intuition • You implement your own SIFT-like descriptor • Ignore scale/orientation to start. • Parameters: stick with defaults + minor tweaks • Feel free to look at papers / resources for inspiration 2017 Spring TA Martin Zhu recommends this tutorial: http://aishack.in/tutorials/sift-scale-invariant-feature-transform-features/ Lowe’s original paper: http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf

  46. Efficient Implementation • Filter using oriented kernels based on directions of histogram bins. • Called ‘steerable filters’

  47. Local Descriptors: SURF • Fast approximation of SIFT idea • Efficient computation by 2D box filters & integral images 6 times faster than SIFT • Equivalent quality for object identification • GPU implementation available • Feature extraction @ 200Hz(detector + descriptor, 640×480 img) • http://www.vision.ee.ethz.ch/~surf [Bay, ECCV’06], [Cornelis, CVGPU’08] K. Grauman, B. Leibe

  48. Local Descriptors: Shape Context Count the number of points inside each bin, e.g.: Count = 4 ... Count = 10 Log-polar binning: More precision for nearby points, more flexibility for farther points. Belongie & Malik, ICCV 2001 K. Grauman, B. Leibe

More Related