Why D.I.P.?

1. Why D.I.P.? • Reasons for compression • Image data need to be accessed at a different time or location • Limited storage space and transmission bandwidth • Reasons for manipulation • Image data might experience nonideal acquisition, transmission or display (e.g., restoration, enhancement and interpolation) • Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) • To produce images with artistic effect (e.g., pointellism) • Reasons for analysis • Image data need to be analyzed automatically in order to reduce the burden of human operators • To teach a computer to “see” in A.I. tasks

2. Beyond Image Processing • The way of thinking • From art (heuristics) to science (principles) • The key is mathematics (I will write a separate blog about the role of mathematics in DIP) • The holistic view • Everything is connected (recall the six-degree phenomenon in social science) • The “Google”-style re-search • Ability to search is a basic part of learning

3. Image Acquisition Image Generation D.I.P. Theme Park Image Compression Image Analysis Image Manipulation Image Display Image Perception DIP is also about connecting dots – in image compression, you will see why you need to learn matrix theory and statistics

4. The Art of Image Compression • Why are images compressible? • Redundancy in images (NOT random) • How data compression works? • Probability theory and statistics • Shannon’s information theory • What about the future of image compression? • I will discuss this in my weblog and facebook (Google “what happened to Iterated Systems Incorporated?”)

5. Shannon’s Picture on Communication (1948) channel encoder channel decoder source channel destination super-channel source encoder source decoder The goal of communication is to move information from here to there and from now to then Examples of source: Human speeches, photos, text messages, computer programs … Examples of channel: storage media, telephone lines, wireless transmission …

6. Lossless vs. Lossy Compression • Lossless: zero error tolerance • No information loss • Shannon’s entropy formula • For photographic images, compression ratio is modest (about 2:1) • Lossy: the goal is to preserve the visual quality of images • Information loss visually acceptable • Shannon’s rate-distortion function • For photographic images, compression ratio is typically around 10-100

7. Popular Lossless Image Compression Techniques • WinZip • Based on the celebrated Lempel-Ziv algorithm • invented nearly 30 years ago • Based on an enhanced version of LZ algorithm • by Welch in 1983 • -Was introduced by CompuServe in 1987 and made • popular until it was not royalty-free in 1994 • GIF (Graphic Interchange Format) • PNG (Portable Network Graphics) GIF Liberation Day: June 20, 2003

8. Lossy Image Compression JPEG decoder compressed JPEG file (20,407 bytes) Q Q decompressed image 100 low compression ratio high quality high compression ratio original raw image (262,144 bytes) low quality 0

9. From JPEG to JPEG2000 discrete cosine transform based wavelet transform based JPEG (CR=64) JPEG2000 (CR=64)

10. Image Acquisition Image Generation D.I.P. Theme Park Image Compression Image Analysis Image Manipulation Image Display Image Perception DIP is also about connecting dots – in image manipulation, you will see why you need to learn calculus and Fourier transform

11. Image Manipulation (I): Noise Removal Noise contamination is often inevitable during the acquisition salt and pepper (impulse) noise additive white Gaussian noise You will learn how to design image filter in a principled way

12. Image Manipulation (II): Deblurring License plate is barely legible due to motion blurring You will learn the use of FT and the necessity of regularization

13. Image Manipulation (III): Contrast Enhancement under-exposed image overly-exposed image You will learn how to modify the histogram of an image

14. Image Manipulation (IV): Aliasing Reduction Example: aliasing artifacts in MRI image acquisition Ideal quality, slow scanning nonideal quality, fast scanning Tradeoff between scanning time and image quality (image reconstruction is covered by EE425)

15. Image Manipulation (V): Image Interpolation digital zooming small 1M pixels large 4M pixels Resolution enhancement can be obtained by common image processing software such as Photoshop or Paint Shop Pro You will learn the difference between digital and optical zooming

16. Image Manipulation (VI): Image Mosaicing Merge multiple images of the same scene into one with larger FOV = + There exist several mosaicing software for automatic stitching F.Y.I.: search “Gigapixel images” by Google http://triton.tpd.tno.nl/gigazoom/Delft2.htm

17. Image Manipulation (VII): Error Concealment blocks contaminated by channel errors (this problem is covered in EE565)

18. Image Manipulation (VIII): Deblocking/Deringing Block artifacts Ringing artifacts You will learn how to suppress those artifacts by nonlinear diffusion

19. Image Manipulation (IX): Dejittering jittering noise (you will see it in either bonus assignment or final project)

20. Image Manipulation (X): Image Inpainting You will learn how to do image inpainting in EE565

21. Image Inpainting Application: Restore Old Photos

22. Image Manipulation (XI): Color Quantization 25,680 colors (24 bits) 256 colors (8 bits) Applications: video cell-phone, gameboy, portable DVD

23. Image Manipulation (XII): Image Halftoning grayscale: 0-255 halftoned: 0/255 You will learn the famous Floyd-Steinberg diffusion in CA

24. Image Manipulation (XIII): Image Scrambling/Hashing* original scrambled

25. Image Manipulation (XIV): Image Watermarking* Original image Modified image

26. Image Manipulation (XV): Image Stylization*

27. Image Manipulation (XVI): Image Rendering* Abyss computer generated

28. Image-based Rendering*

29. Image Acquisition Image Generation D.I.P. Theme Park Image Compression Image Analysis Image Manipulation Image Display Image Perception DIP is also about connecting dots – in image analysis, you will see why you need to know about neuroscience and psychology

30. Image Analysis (I): Edge Detection You will learn basic edge detectors based on derivatives

31. Image Analysis (II): Face Detection Deceivingly simple for humans but notoriously difficult for machines

32. Image Analysis (III): Change Detection

33. Change Detection in Medical Application

34. Image Analysis (IV): Image Matching Antemortem dental X-ray record Postmortem dental X-ray record

35. Image Matching in Biometrics Two deceivingly similar fingerprints of two different people

36. Image Analysis (V): Image Segmentation

37. Image Analysis (VI): Object Recognition License number can be automatically extracted from the image of license plate

38. Object Recognition in Military Applications

39. Image Analysis (VII): Event Recognition Image-based monitoring system prevents drowning

40. Image Analysis (VIII): Video Summarization Only send out “important” motion pictures such as home-runs

41. Image Analysis (IX): Content-based Image Retrieval retrieved building images

42. Summary • In EE465, you will learn • Image compression: Lempel-Ziv, Huffman coding, run-length coding and JPEG • Image manipulation: linear/nonlinear filtering, histogram-based processing, linear interpolation • Image analysis: edge/corner detection, circle/ line/ellipse detection, chain codes/shape numbers • In EE565, you will learn • Advanced algorithms/techniques with stronger mathematical emphasis • Not covered by the courses I offered • CBIR (multimedia database), face/pedestrian detection (Advanced biometrics), PDE-based image processing (medical image analysis)

43. General Cooking Recipe • Motivations • Problem statement • Heuristic observations • Examples/Illustrations • Principled approaches (via mathematics) • MATLAB implementations • Computer assignment reviews • Current state-of-the-art and future directions