1 / 48

Statistical Image Quality Measures

Statistical Image Quality Measures. Hiroyuki Takeda, Hae Jong Seo, Peyman Milanfar EE Department University of California, Santa Cruz. Jan 11, 2008. Overview. Background. CCA-based Similarity Measure (Full-reference). SVD-based Quality Measure (No-reference). Conclusion. Slide 1.

bevis
Download Presentation

Statistical Image Quality Measures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical Image Quality Measures Hiroyuki Takeda, Hae Jong Seo, Peyman Milanfar EE Department University of California, Santa Cruz Jan 11, 2008

  2. Overview Background CCA-based Similarity Measure (Full-reference) SVD-based Quality Measure (No-reference) Conclusion Slide 1

  3. Objective Quality Assessment Develop quantitative measures that automatically predict the perceived image quality Full-reference No-reference Reduced-reference Applications Image acquisition, compression, communication, displaying, printing, restoration Slide 2

  4. Overview Background CCA-based Similarity Measure (Full-reference) SVD-based Quality Measure (No-reference) Conclusion

  5. Full-Reference Image Quality Measure Structural Similarity Measure [1] Focus on perceived changes in structural informationvariation unlike error based approach ( i.e. MSE or PSNR ) MSE : 210 Mean shifted Blurred JPEG compressed Contrast stretched Salt-pepper Original image [1] Zhou Wang et al,  “Image Quality Assessment: From Error Visibility to Structural Similarity ”, IEEE TIP ‘ 04 Slide 3

  6. Structural Similarity Measure Three components : Luminance , Contrast , Structure Small constant Image patches being compared Slide 4

  7. Drawback of SSIM SSIM: 0.505 SSIM: 0.549 SSIM: 0.551 Original Zoom Out Translation Rotation Sensitive to spatial translation, rotation, and scale changes due to simple correlation coefficient Solution A powerful statistical tool : Canonical Correlation Analysis (Hotelling, 1936) Slide 5

  8. New Statistical Image Quality Measure Canonical Correlation Analysis (CCA) : Find out a pair of direction vectors which maximally correlate the two datasets : canonical correlation : Useful property Affine–invariance Slide 6

  9. New Statistical Image Quality Measure Canonical Correlation Structural Similarity Measure : Local Search Window at i th position P : Pixel intensity Gx,Gy : Gradients A B CCA P P CCA Gx Gy Gx Gy CCA Gx Gy P Gx Gy P Slide 7

  10. New Statistical Image Quality Measure Mathematical Solution 1) Calculate Covariance Matrix 2) Solve coupled eigen-value problems 3) Define CCSIM as largest canonical correlation Slide 8

  11. Examples (1) Original Image Zoom Out 1 2 Slide 9

  12. Examples (2) Original Image 1 2 1 0.34 0.73 Zoom Out 2 SSIM CCSIM Slide 10

  13. Examples (2) Original Image Translation 1 3 Slide 11

  14. Examples (2) Original Image 1 3 1 0.38 0.75 Translation 3 SSIM CCSIM Slide 12

  15. Examples (3) Original Image Rotation 1 4 Slide 13

  16. Examples (3) Original Image 1 4 1 0.41 0.77 Rotation 4 SSIM CCSIM Slide 14

  17. JPEG Compression Example Clean image (QF=100) JPEG(QF=50) JPEG(QF=10) 1 2 3 8 bits/pixel 0.899 bits/pixel 0.352 bits/pixel Slide 15

  18. JPEG Compression Example 0.90 0.79 Clean Image 1 1 1 2 3 JPEG (QF =50) 2 SSIM SSIM 0.85 0.79 JPEG (QF =10) 3 CCSIM CCSIM Slide 16

  19. Clean Image VS Compressed Images Quality SSIM CCSIM JPEG quality factor Slide 17

  20. Denoising Example Clean Image WGN(sigma=15) Denoised by SKR[2] 1 2 3 [2] Takeda et al.,  “ Kernel Regression for image processing and reconstruction ”, IEEE TIP ‘ 07 Slide 18

  21. Denoising Example Clean Image 0.47 0.89 1 1 1 2 3 WGN( sigma =15 ) 2 SSIM SSIM 0.47 0.89 Denoised by SKR 3 CCSIM CCSIM Slide 19

  22. Clean VS (Noisy & Denoised images) Clean VS Noisy Clean VS Denoised Quality Quality CCSIM SSIM SSIM CCSIM WGN: Noise level WGN: Noise level Slide 20

  23. Super-resolution Motion Estimation Steering Kernel Regression Resolution enhancement from video frames captured by a commercial webcam(3COM Model No.3719) Slide 21

  24. Super-resolution Example Clean Image (512 x 512) Low resolution Sequence (64x64 32 frames) Super-resolved by SKR 1 2 3 Slide 22

  25. Super-resolution Example Clean Image 1 3 1 0.87 0.91 Low resolution Sequence( 32 frames) 2 SSIM CCSIM Super-resolved by SKR 3 Slide 23

  26. Overview Background CCA-based Similarity Measure (Full-reference) SVD-based Quality Measure (No-reference) Conclusion

  27. No-Reference SVD-Based Measure Singular value decomposition of local gradient matrix: N x N SVD Local orientation dominance It becomes close to 1 when there is one dominant orientation in a local area. It takes on small values in flat or highly textured (or pure noise) area. So, this quantity tells us about the “edginess” of the region being examined.

  28. Properties of Local Orientation Dominance(1) Density function for i.i.d. white Gaussian noise N: the window size N=11 Note : the PDF is independent from the noise variance, but depends on the window size. N=9 N=7 N=5 [1] A. Edelman. Eigenvalues and condition numbers of random matrices, SIAM Journal on Matrix Analysis and Applications 9 (1988), 543-560. N=3 [2] X. Feng and P. Milanfar. Multiscale principal component Analysis for Image Local Orientation Estimation, Proceeding of 36th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov. 2002 Slide 25

  29. Properties of Local Orientation Dominance(2) The mean values for a variety of test images with added white Gaussian noise. N = 11 The mean values for pure noise are always constant. 0.06 Remember the number Slide 26

  30. The Performance Analysis Suppose we have a noisy image and a denoised version using some filter: : a given noisy image : the estimated (denoised) image : the residual image If the filter cleans up the given image effectively, The residual image is essentially just noise. of the residual image must be close to the value expected for pure noise. Slide 27

  31. Example (1) Image denoising by bilateral filter Bilateral filter has two parameters: Spatial smoothing parameter , and radiometric smoothing parameter Denoising experiment A noisy image, Added white Gaussian noise, SNR=20dB, PSNR=29.25dB, RMSE = 8.67 The original image C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images”, Proceedings of the 1998 IEEE International Conference of Computer Vision, Bombay, India, pp. 836-846, January 1998. Slide 28

  32. The Performance Analysis of Bilateral Filter The plot of as a function of the smoothing parameters: N = 11 Slide 29

  33. Denoising Result The noisy image Bilateral filter PSNR = 42.87dB, RMSE = 1.833 Residual Slide 30

  34. The Performance Analysis of Bilateral Filter The plot of as a function of the smoothing parameters: N = 11 Slide 31

  35. Denoising Result The filter also removes image contents. Bilateral filter PSNR = 39.57dB RMSE = 2.68 The noisy image Residual Slide 32

  36. What If We Pick the Parameters by the Best RMSE? The plot of RMSE as a function of the smoothing parameters: Slide 33

  37. Denoising Result Bilateral filter, PSNR = 42.87dB RMSE = 1.832 The noisy image Residual Slide 34

  38. Example (2) Iterative Steering Kernel Regression Iteratively cleaning up noisy images Using the local orientation dominance, we find the optimal number of iterations. The original image The noisy image, Added white Gaussian noise, SNR=5.6dB, PSNR = 20.22dB RMSE = 24.87 Slide 35

  39. Denoising Result (1) The plot of as a function of the smoothing parameters: Slide 36

  40. Denoising Result ISKR, IT = 15, PSNR = 31.33 dB RMSE = 6.92 The noisy image Residual Slide 37

  41. If the Ground Truth is Available, The plot of RMSE as a function of the smoothing parameters: RMSE Slide 38

  42. Denoising Result ISKR, IT = 12, PSNR = 31.69 dB RMSE = 6.64 The noisy image Residual Slide 39

  43. Overview Background CCA-based Similarity Measure (Full-reference) SVD-based Quality Measure (No-reference) Conclusion

  44. Conclusion Two new statistical quality measures CCSIM(CCA-based) : full-reference SVD-based measure: no-reference CCSIM is a general version of SSIM We showed examples of JPEG compression, denoising , and super Resolution with comparison to SSIM SVD-based measure is applicable for any denoising filter. We illustrated application to global parameter optimization. Locally adaptive parameter optimization is also possible. The proposed methods can be easily extended to video using 3-d local window. Slide 40

  45. Authors [1] Hiroyuki Takeda : hiro@soe.ucsc.edu www.ucsc.edu/~htakeda [2] Hae Jong Seo : rokaf@soe.ucsc.edu www.ucsc.edu /~rokaf [3] Peyman Milanfar : milanfar@soe.ucsc.edu www.ucsc.edu/~milanfar

  46. Thank you !

  47. Super-resolution Example Down-sampled(2) +WGN(sigma=15) Super-resolved by SKR Clean Image 1 2 3 Extra 1

  48. Super-resolution Example Clean Image 1 1 3 0.71 0.85 Down-sampled(2) +WGN (sigma=15) 2 SSIM CCSIM Super-resolved by SKR 3 Extra 2

More Related