3D Computer Vision: CSc 83020

1. 3D Computer Vision: CSc 83020

2. 3D Computer Vision: CSc 83020 • Instructor: Ioannis Stamos • istamos (at) hunter.cuny.edu • http://www.cs.hunter.cuny.edu/~ioannis • Office Hours: Tuesdays 4-6 (at Hunter) or by appoitnment • Office: 1090G Hunter North (69th street bw. Park and Lex.) • Computer Vision Lab: 1090E Hunter North • Course web page: http://www.cs.hunter.cuny.edu/~ioannis/3D_f12.html

3. Goals • To familiarize you with basic the techniques and jargon in the field • To enable you to solve computer vision problems • To let you experience (and appreciate!) the difficulties of real-world computer vision • To get you excited!

4. Class Policy • You have to • Turn in all assignments (60% of grade) • Complete a final project (30% of grade) • Actively participate in class (10% of grade) • Late policy • Six late days (but not for final project) • Teaming • For final project you can work in groups of 2

5. About me • 11th year at Hunter and the Graduate Center • Graduated from Columbia in ’01 • CS Ph.D. • Research Areas: • Computer Vision • 3D Modeling • Computer Graphics

6. Books Computer Vision: Algorithms and Applications, Richard Szeliski, 2010 (available online for free) Robot Vision B. K. P. Horn, The MIT Press (great classic book) Introductory Techniques for 3-D Computer Vision Emanuele Trucco and Alessandro Verri, Prentice Hall, 1998 (algorithmic perspective) Computer Vision A Modern Approach David A. Forsyth, Jean Ponce, Prentice Hall 2003 An Invitation to 3-D Vision Yi Ma, Stefano Soatto, Jana Kosecka, S. Shankar Sastry Springer 2004. Three-Dimensional Computer Vision: A Geometric Viewpoint Olivier Faugeras The MIT Press, 1996.

7. Journals/Web • International Journal of Computer Vision. • Computer Vision and Image Understanding. • IEEE Trans. on Pattern Analysis and Machine Intelligence. • SIGGRAPH (mostly Graphics) • http://www.ri.cmu.edu/ (CMU’s Robotic Institute) • http://www.cs.cmu.edu/~cil/vision.html (The Vision Home Page) • http://www.dai.ed.ac.uk/CVonline/ (CV Online) • http://iris.usc.edu/Vision-Notes/bibliography/contents.html (Annotated CV Bibliography)

8. Class History • Based on class taught at Columbia University by Prof. Shree Nayar. • New material reflects modern approach. • Taught similar class at Hunter • Taught “3D Photography” class at the Graduate Center of CUNY. • My active research area • Funded by the National Science Foundation

9. Class Schedule • Check class website • Final project proposals • Due Nov. 7 • Design your own or check list of possible projects on class website • Final project presentations and report • May 16 (last class)

10. What is Computer Vision? Sensors Images or Video Illumination Vision System Physical 3D World Scene Description Measuring Visual Information

11. Computer Graphics Output Image Model Synthetic Camera (slides courtesy of Michael Cohen)

12. Computer Vision Output Model Real Scene Real Cameras (slides courtesy of Michael Cohen)

13. Combined Output Image Real Scene Model Synthetic Camera Real Cameras (slides courtesy of Michael Cohen)

14. Cont. • Vision is automating visual processes (Ball & Brown). • Vision is an information processing task (Marr). • Vision is inverting image formation (Horn). • Vision is inverse graphics. • Vision looks easy, but is difficult. • Vision is difficult, but it is fun (Kanade). • Vision is useful.

15. Some Applications • Industrial • Material Handling • Inspection • Assembly

16. Some Applications Autonomous Navigation

17. Some Applications Vision for Graphics Film Industry Urban Planning E-commerce Virtual Reality

18. Some Applications • Realistic 3D experience • Google Earth http://earth.google.com/ • Microsoft Photosynth http://labs.live.com/photosynth/

19. More Applications! • Optical Character Recognition (OCR) • Visual Databases (images or movies) • Searching for image content • Face Recognition (security) • Iris Recognition (security) • Traffic Monitoring Systems • Many more…

20. Vision deals with images

21. Images Look Nice…

22. Images Look Nice… Ioannis Stamos – CSc 83020 Spring 2007

23. ...Essentially a 2D array of numbers 107 132 107 107 132 99 132 107 132 99 107 132 99 107 132 91 107 132 99 132 99 107 107 132 99 132 107 132 107 132 91 107 132 107 132 99 107 132 107 132 107 99 132 99 132 99 132 99 132 124 132 99 132 107 132 132 107 132 124 132 132 124 132 150 107 150 150 132 150 132 150 132 150 107 150 132 124 132 132 150 107 99 132 132 107 132 107 132 150 132 150 99 132 107 150 132 107 150 132 124 132 132 107 150 99 150 107 150 132 107 150 132 124 132 150 115 124 132 150 107 132 150 132 150 150 107 132 116 132 124 132 107 99 150 132 107 132 150 132 124 132 150 107 150 107 132 99 132 107 150 132 150 107 150 132 150 150 107 107 150 150 150 150 115 167 107 150 107 132 150 107 150 132 124 132 124 132 124 132 124 132 150 107 150 107 107 132 116 132 150 132 150 107 150 150 132 150 132 116 132 124 132 150 132 150 150 150 132 116 132 116 107 132 99 150 150 132 107 132 150 107 150 132 124 132 116 132 107 150 132 107 150 132 150 107 150 107 132

24. Low-Level or “Early” Vision • Considers local properties of an image “There’s an edge!” From: Szymon Rusinkiewicz, Princeton.

25. Mid-Level Vision • Grouping and segmentation “There’s an object and a background!”

26. High-Level Vision • Recognition “It’s a chair!”

27. Humans Vision is easy for us. But how do we do it?

28. Human Vision: Illusions Fraser’s spiral (Fraser 1908)

29. Illusions Zölner Illusion (1860) Hering Illusion (1861) Wundt Illusion (1896)

30. Visual Ambiguities Young-Girl/Old-Woman

31. Visual Ambiguities

32. Visual Ambiguities

33. Seeing and Thinking Kanizsa (1979)

34. Syllabus Overview

35. Image Formation and Optics Light Source p Surface normal CCD Array Lens P Object Surface Projection of 3-D World on a 2-D plane

36. Lenses Ray of light Optical Axis

37. Image Sensors/Camera Models Typical 512x512 CCD array Imaging Area 262,144 pixels One Pixel 20μm 20μm 512 (10.25mm) Convert Optical Images To Electrical Signals. 512 (10.25mm)

38. Filtering = h g f

39. Image Features Detecting intensity changes in the image Ioannis Stamos – CSc 83020 Spring 2007

40. Grouping image features Finding continuous lines from edge segments Ioannis Stamos – CSc 83020 Spring 2007

41. Camera Calibration Camera Coordinate Frame Zc Pixel Coordinates Yc Intrinsic Parameters Xc Extrinsic Parameters Image Coordinate Frame Zw Yw World Coordinate Frame Xw

42. Shape from X • Shape from X • Stereo • Motion • Shading • Texture foreshortening

43. Binocular Stereo depth map

44. Active Sensing Sources of error: 1) grazing angle, 2) object boundaries. Sheet of light CCD array Lens

45. Shape from Shading Three-dimensional shape from a single image. Ioannis Stamos – CSc 83020 Spring 2007

46. Motion (optical flow) Determining the movement of scene objects Ioannis Stamos – CSc 83020 Spring 2007

47. Reflectance and Color Why do these spheres look different?

48. Object Recognition Learning visual appearance. Real-time object recognition.

49. Template-Based Methods Cootes et al.

50. Some Vision Systems…