1 / 12

Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on 

Motion vector processing based on residual energy information for motion compensated frame interpolation. Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on . introduction. Low bandwidth requirements, i.e. video telephony by skipping frames

Download Presentation

Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on 

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Motion vector processing based on residual energy information formotion compensated frame interpolation Ai-Mei Huang And Truong Nguyen Image processing, 2006 IEEE international conference on 

  2. introduction • Low bandwidth requirements, i.e. video telephony • by skipping frames • low frame rate video usually results in motion jerkiness • Motion-compensated frame interpolation (MCFI) • Directly uses the received MVs • Suffer from annoying artifacts such as blockiness and ghost effect. • Motion re-estimation is not suited for mobile devices Video coding/compression Video in encoder Bit stream Reconstructed frame decoder

  3. Motion compensated frame interpolation • In MCFI methods, a skipped frame is often bi-directionally interpolated • from its two neighboring reconstructed frames . • by using the received MVs of the second frame. MV is not always reliable!

  4. The correlation between motion vectorreliability and residual energy. • Macroblock(MB) contain areas of different motion. • Encoder favors the MV that can represent most of the region • The prediction residual will be generated and encoded. • Correct those unreliable MVs for frame interpolation • Avoid using those unreliable MVs to correct other MVs.

  5. The proposed method • Based on residual energy associated with each motion vector • iterative process, stops when the MAP is no longer changed, or reaches a pre-defined maximum iteration, max_ite.

  6. (1)motion vector classification based on residual energy (base unit) 16*16 MB=>four 8*8 blocks • Classify MVs to 3 Groups. • Calculate every 8x8 block’s residual energy Em,n • By taking the sum of the absolute value of each reconstructed prediction error for each pixel. • Em,n<threshold  MV will be classified as reliable and place into first group G1. • Em,n>=threshold  MV will be classified as unreliable and place into first group G3. • Unreliable MV’s neighboring MVs within the same MB will be classified as possibly unreliable into the second group G2

  7. (2) motion vector correction • Works on those unreliable MVs Correcting from neighbor reliable MV • A residual-energy constrained median filter (RECMF) is used in this process and defined as • S contains the neighboring MVs centered at v*m,n. • Select a new MV only from its neighboring reliable MVs. • vm,n itself will be excluded from the candidates. • we prevent unreliable MVs to be used to correct other unreliable MVs.

  8. (3)motion vector similarity check • Ensure v*m,n is not identical or similar to vm,n • Remain unreliable • Use angle variance by taking inner product of the two vectors. • < threshold => two MVs is similar ,fail in the check => bm,n still in G3 • Pass in the check => bm,n take in G1

  9. (4)motion vector re-sampling and smoothing • In [7], each 8 x 8 block is further broken into four 4 x4 sub-blocks. • The MVs of these four sub-blocks can be obtained simultaneously by minimizing a smoothness measure , which is defined in the following. • For example: [7] G. Dane and T. Q. Nguyen, "Smooth motion vector resampling for standard compatible video post-processing," Proc. Asilomar Conf: Signals, Systems and Computers, 2004.

  10. SIMULATIONS • two video sequences, FOREMAN and SILENT, • CIF frame resolution • 30 frame per second (fps). • both encoded using H.263 but even frames are skipped by the encoder. • fix quantization parameter (QP) values. • Avg. bitrate • FOREMAN is 240.3 Kbps • SILENT is184.59 Kbps

More Related