Multi-Perspective Panoramas

1. Multi-Perspective Panoramas CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2017 Slides from Lihi Zelnik

2. Objectives • Better looking panoramas • Let the camera move: • Any view • Natural photographing

3. Stand on the shoulders of giants Cartographers Artists

4. Cartographic projections

5. φ θ Common panorama projections Perspective Stereographic Cylindircal

6. Global Projections Perspective Stereographic Cylindircal

7. Sharp discontinuity perspective perspective Learn from the artists Multiple view points De Chirico “Mystery and Melancholy of a Street”, 1914

8. Renaissance painters solution “School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

9. Personalized projections “School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!

10. Multiple planes of projection Sharp discontinuities can often be well hidden

11. Single view multi-view result

12. Single view multi-view result

13. Single view multi-view result

14. Single view multi-view result

15. Objectives - revisited • Better looking panoramas • Let the camera move: • Any view • Natural photographing Multiple views can live together

16. Multi-view compositions 3D!! David Hockney, Place Furstenberg, (1985)

17. Why multi-view? Multiple viewpoints Single viewpoint David Hockney, Place Furstenberg, 1985 Melissa Slemin, Place Furstenberg, 2003

18. Long Imaging Agarwala et al. (SIGGRAPH 2006)

19. Smooth Multi-View Google maps

20. What’s wrong in the picture? Google maps

21. Non-smooth Google maps

22. The Chair David Hockney (1985)

23. Joiners are popular Flickr statistics (Aug’07): 4,985 photos matching joiners. 4,007 photos matching Hockney. 41 groups about Hockney Thousands of members

24. Main goals: Automate joinersGeneralize panoramas to general image collections

25. Objectives • For Artists:Reduce manual labor Fully automatic Manual: ~40min.

26. Objectives • For Artists:Reduce manual labor • For non-artists:Generate pleasing-to-the-eye joiners

27. Objectives • For Artists:Reduce manual labor • For non-artists:Generate pleasing-to-the-eye joiners • For data exploration:Organize images spatially

28. What’s going on here?

29. A cacti garden

30. Principles

31. Principles • Convey topology Correct Incorrect

32. Principles • Convey topology • A 2D layering of images Blending: blurry Graph-cut: cuts hood Desired joiner

33. Principles • Convey topology • A 2D layering of images • Don’t distort images translate rotate scale

34. Principles • Convey topology • A 2D layering of images • Don’t distort images • Minimize inconsistencies Bad Good

35. Algorithm

36. Step 1: Feature matching Brown & Lowe, ICCV’03

37. Step 2: Align Large inconsistencies Brown & Lowe, ICCV’03

38. Step 3: Order Reduced inconsistencies

39. Ordering images Try all orders: only for small datasets

40. Ordering images Try all orders: only for small datasets complexity: (m+n)m = # imagesn = # overlaps = # acyclic orders

41. Ordering images Observations: • Typically each image overlaps with only a few others • Many decisions can be taken locally

42. Ordering images Approximate solution: • Solve for each image independently • Iterate over all images

43. Can we do better?

44. Step 4: Improve alignment

45. Iterate Align-Order-Importance

46. Iterative refinement Initial Final

47. Iterative refinement Initial Final

48. Iterative refinement Initial Final

49. What is this?

50. That’s me reading