1 / 96

Motion estimation

Motion estimation. Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/3/23. with slides by Michael Black and P. Anandan. Announcements. Project #1 is due on next Tuesday, submission mechanism will be announced later this week.

jonesdavid
Download Presentation

Motion estimation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Motion estimation Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/3/23 with slides by Michael Black and P. Anandan

  2. Announcements • Project #1 is due on next Tuesday, submission mechanism will be announced later this week. • grading: report is important, results (good/bad), discussions on implementation, interface, features, etc.

  3. Outline • Motion estimation • Lucas-Kanade algorithm • Tracking • Optical flow

  4. Motion estimation • Parametric motion (image alignment) • Tracking • Optical flow

  5. Parametric motion

  6. Tracking

  7. Optical flow

  8. Three assumptions • Brightness consistency • Spatial coherence • Temporal persistence

  9. Brightness consistency

  10. Spatial coherence

  11. Temporal persistence

  12. Image registration Goal: register a template image J(x) and an input image I(x), where x=(x,y)T. Image alignment: I(x) and J(x) are two images Tracking: I(x) is the image at time t. J(x) is a small patch around the point p in the image at t+1. Optical flow: I(x) and J(x) are images of t and t+1.

  13. Simple approach • Minimize brightness difference

  14. Simple SSD algorithm For each offset (u, v) compute E(u,v); Choose (u, v) which minimizes E(u,v); Problems: • Not efficient • No sub-pixel accuracy

  15. Lucas-Kanade algorithm

  16. Newton’s method • Root finding for f(x)=0 Taylor’s expansion:

  17. Lucas-Kanade algorithm

  18. Lucas-Kanade algorithm

  19. Lucas-Kanade algorithm iterate shift I(x,y) with (u,v) compute gradient image Ix, Iy compute error image J(x,y)-I(x,y) compute Hessian matrix solve the linear system (u,v)=(u,v)+(∆u,∆v) until converge

  20. translation affine Parametric model

  21. minimize Parametric model minimize with respect to

  22. warped image image gradient Jacobian of the warp Parametric model

  23. Jacobian of the warp For example, for affine

  24. Parametric model minimize Hessian

  25. Lucas-Kanade algorithm iterate warp I with W(x;p) compute error image J(x,y)-I(W(x,p)) compute gradient image evaluate Jacobian at (x;p) compute compute Hessian compute solve update p by p+ until converge

  26. Coarse-to-fine strategy I J refine J Jw I warp + I J Jw pyramid construction pyramid construction refine warp + J I Jw refine warp +

  27. Application of image alignment

  28. Tracking

  29. Tracking

  30. Tracking brightness constancy optical flow constraint equation

  31. Optical flow constraint equation

  32. Multiple constraint

  33. Area-based method • Assume spatial smoothness

  34. Aperture problem

  35. Aperture problem

  36. Aperture problem

  37. Demo for aperture problem • http://www.sandlotscience.com/Distortions/Breathing_objects.htm • http://www.sandlotscience.com/Ambiguous/barberpole.htm

  38. Aperture problem • Larger window reduces ambiguity, but easily violates spatial smoothness assumption

  39. Area-based method • Assume spatial smoothness

  40. Area-based method must be invertible

  41. Area-based method • The eigenvalues tell us about the local image structure. • They also tell us how well we can estimate the flow in both directions • Link to Harris corner detector

  42. Textured area

  43. Edge

  44. Homogenous area

  45. KLT tracking • Select feature by • Monitor features by measuring dissimilarity

  46. KLT tracking http://www.ces.clemson.edu/~stb/klt/

More Related