CS 585 Computational Photography

1. CS 585 Computational Photography Nathan Jacobs

2. Overview • We can think of images as vectors • From linear algebra, we can represent images in alternative bases • One really useful basis is based on frequency domain content

3. Salvador Dali “Gala Contemplating the Mediterranean Sea, which at 30 meters becomes the portrait of Abraham Lincoln”, 1976 Salvador Dali, “Gala Contemplating the Mediterranean Sea, which at 30 meters becomes the portrait of Abraham Lincoln”, 1976 Salvador Dali, “Gala Contemplating the Mediterranean Sea, which at 30 meters becomes the portrait of Abraham Lincoln”, 1976

6. Why is this interesting? • Information in low frequencies (blurry version of the image) • Information in high frequencies (local changes • Motivates alternative representations of images… not just an array of pixel values.

7. Digression: Images and Linear Algebra • Vectors are represented in a basis In linear algebra, a basis is a set of linearly independentvectors that, in a linear combination, can represent every vector in a given vector space… Given a basis of a vector space, every element of the vector space can be expressed uniquely as a finite linear combination of basis vectors. Every vector space has a basis, and all bases of a vector space have the same number of elements, called the dimension of the vector space. (linear algebra) (from wikipedia) Standard basis is the identity matrix… but any lin. independent matrix will do.

8. Digression: Images and Linear Algebra • Images are vectors… = = … Any matrix A that is linearly independent… …

9. Jean Baptiste Joseph Fourier (1768-1830) • had crazy idea (1807): • Any periodic function can be rewritten as a weighted sum of sines and cosines of different frequencies. • Don’t believe it? • Neither did Lagrange, Laplace, Poisson and other big wigs • Not translated into English until 1878! • But it’s true! • called Fourier Series

10. A sum of sines • Our building block: • Add enough of them to get any signal f(x) you want!

11. Inverse Fourier Transform Fourier Transform F(w) f(x) F(w) f(x) Fourier Transform • We want to understand the frequency w of our signal. So, let’s reparametrize the signal by w instead of x: • For every w from 0 to inf, F(w) holds the amplitude A and phase f of the corresponding sine • How can F hold both? Complex number trick! We can always go back:

12. Fourier Transform: a special basis matrix

13. Time and Frequency • example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t)

14. Time and Frequency • example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t) = +

15. Frequency Spectra • example : g(t) = sin(2pf t) + (1/3)sin(2p(3f) t) = +

16. Frequency Spectra • Usually, frequency is more interesting than the phase

17. Frequency Spectra = + =

18. Frequency Spectra = + =

19. Frequency Spectra = + =

20. Frequency Spectra = + =

21. Frequency Spectra = + =

22. Frequency Spectra =

23. Frequency Spectra

24. Extension to 2D in Matlab, check out: imagesc(log(abs(fftshift(fft2(im)))));

25. Man-made Scene

26. Low and High Pass filtering

27. Can change spectrum, then reconstruct

28. Relationship to JPEG Image Compression

29. Moving convolution from n2 to n log n. The Fourier transform of the convolution of two functions is the product of their Fourier transforms The inverse Fourier transform of the product of two Fourier transforms is the convolution of the two inverse Fourier transforms Convolution in spatial domain is equivalent to multiplication in frequency domain! The Convolution Theorem

30. 2D convolution theorem example |F(sx,sy)| f(x,y) * h(x,y) |H(sx,sy)| g(x,y) |G(sx,sy)| Note: this is like the high-pass filter from earlier slide

31. Fourier Transform pairs

32. Conclusion • Images are vectors • Fourier transform… a special image basis • Convolution theorem: faster convolutions, really helps on large filters… actually slower on small sizes • on my laptop, breakeven at roughly 50x50

33. Proof in Linear Algebrahttp://en.wikipedia.org/wiki/Convolution_theorem • The proof is trivial in linear algebra where convolution is represented by an infinite-dimensional Toeplitz matrix, h, which are known to have the Fourier eigenbasis, F. This means that h can be represented by a diagonal one, H = (F h F-1),