Temporal Video Denoising Based on Multihypothesis Motion Compensation

1. Temporal Video Denoising Based on Multihypothesis Motion Compensation LiweiGuo; Au, O.C.; Mengyao Ma; Zhiqin Liang; Hong Kong Univ. of Sci. & Technol., Clear Water Bay Circuits and Systems for Video Technology(CSVT), IEEE 2007

2. Outline • Introduction • Video Signal Model With Multihypothesis MC • MultihypothesisMotion Compensated Filter (MHMCF) • The Proposed Linear Temporal Filter – MHMCF • Implementation Issues • Performance Analysis • Experimental Results • Conclusions

3. Introduction • Spatial correlation denoising: 2-D Kalman filter [1], 2-D Wiener filter [2], wavelet shrinkage [3], non-local means [4] etc. • Until now there are few temporal denoisingmethods presented in the literature. • These temporal predictions, defined as its motion-compensatedhypotheses for the current pixel.

4. Video Signal Model With MultihypothesisMC • We present a novel model of residue(zm)for multihypothesis MC: • Let the mean and the varianceof be and respectively. • We propose a linear model for this relationship: f:the current pixel of Fk Pm:the motion compensated prediction of f from Fk-m

5. Video Signal Model With MultihypothesisMC

6. Video Signal Model With MultihypothesisMC • For video with large motion, the correlation tends to decrease faster than small motion. • Large b implies video with large motion. • Large a implies texture regions.

7. Multihypothesis Motion Compensated Filter (MHMCF) • The Proposed Linear Temporal Filter – MHMCF • Implementation Issues • Motion Estimation • Parameters Estimation • Performance Analysis

8. -The Proposed Linear Temporal Filter – MHMCF • Assumptions: • Video sequence is contaminated by additive zero-mean random noise. • The noise source is stationaryover spatial and temporaldomain, and independent of residue(zm). • The noise-corrupted video signal f’ and p’m: • We propose MHMCF to estimate the current pixel f:

9. -The Proposed Linear Temporal Filter – MHMCF • For simplicity, we rewrite (3) as: • We define the objective function of MHMCF: • Minimizing is equal to Minimizing [16, p. 273]. Random varianbles Let

10. -The Proposed Linear Temporal Filter – MHMCF • By = 0 and = 0 , the optimal wanddthat minimize are: • As zm and nmare independent, and gmisindependent with each other:

11. -The Proposed Linear Temporal Filter – MHMCF • The optimal w and d that minimize are: • Large implies low temporal correlation. • When , then d = 0, w0 = 1, wm = 0, and no filtering will be applied.

12. -Implementation Issues • Motion Estimation: • MHMCF needs to performME with respect to every reference frame. • Fast ME algorithm, PMVFAST [17], is employed. • Experiments show that PMVFASTcompared to full search, about 1% denoising error is increased. • Parameters Estimation: ?

13. -Implementation Issues • Parameters Estimation: • : We select the minimum 3% out of the total block variances (their average is ) : • and : Let be the noisy residue. • Then ( ), since n0 and nmare all zero-mean: • As gm and n0 are independent:

14. -Performance Analysis • The estimation error : • MHMCF is an unbiased estimator leading to and error variance . • Combining (3), (6), (9),(11), and (12): • We have the model of residue variance . • The remainingnoise in the reference frameis the estimation error: ?

15. -Performance Analysis Smooth regions Small motion

16. -Performance Analysis

17. Experimental Results • DenoisingPerformance: JNT[9] STVF[10] WRSTF[11]

18. Experimental Results

19. Experimental Results

20. Experimental Results • Computational Complexity : • In terms of the number of ADD and MUL performed to process a frame in CIF resolution (352 288).

21. Conclusions • A temporal denoising filter MHMCF is developed for the removal of noise in video. • MHMCF has very good noise suppression capability while using fewer inputs than other proposed filters. • MHMCF is a purely temporal filter, spatial blurring is avoided and most spatial details could be preserved.