1 / 27

Detection and Removal of Rain from Videos

Detection and Removal of Rain from Videos Kshitiz Garg and Shree K. Nayar Department of Computer Science Columbia University IEEE CVPR Conference June 2004, Washington DC, USA Sponsors: NSF, DARPA Outdoor Vision and Weather Fog Mist [Nayar and Narasimhan,99] [Narasimhan and Nayar,

ostinmannual
Download Presentation

Detection and Removal of Rain from Videos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Detection and Removal of Rain from Videos Kshitiz Garg and Shree K. Nayar Department of Computer Science Columbia University IEEE CVPR Conference June 2004, Washington DC, USA Sponsors: NSF, DARPA

  2. Outdoor Vision and Weather Fog Mist [Nayar and Narasimhan,99] [Narasimhan and Nayar, 00,01,02,03] Rain Snow

  3. Steady versus Dynamic Weather Steady Weather (Fog, Mist) Dynamic Weather (Rain, Snow) Pixel Pixel Fog-droplet (1-10) Raindrop (0.5-10) Aggregate scattering Individual scattering Steady Weather Dynamic Weather Objects

  4. Previous Work on Rain • Atmospheric Sciences • Physical Properties of Rain • Communication • Signal Transmission through Rain • Graphics • Particle Systems • Heuristic Based Methods [Beard and Chung 1987, Wang 1975, Marshall and Palmer 1948,Mason 1975] [ Ting 1982,Manning 1993, Robert 1992 ] [ Reeves 1983, Sims 1997 ] [ Starik and Werman 2003 ]

  5. Median Filtering Image Processing Does Not Suffice Rain with Scene Motion Median Filtered Video Courtesy © New Line Productions, Inc. Difference (Original and Median Filtered)

  6. Modeling the Visual Effects of Rain Rain Drop Rain Streak Dynamics Rain Photometry

  7. 800 600 2.5 mm Number Density 2 400 1.5 1 0.5 Falling Direction 200 4 1 2 3 a Drop radius (mm) Shapes of Rain Drop Size Distribution of Rain Drop Physical Properties of Rain Drops Beard and Chung 1987 Marshall and Palmer 1948 (1/m) N Drop density 95% of Rain Drops are Spherical

  8. Visual Appearance of a Rain Drop Beard and Chung 1987 Marshall and Palmer 1948 Specular Reflection N Refraction Internal Reflection

  9. Brightness Properties of a Rain Drop Refraction Specular 1 0.8 FOV=1650 94% 0.6 0.4 0.2 Internal Reflection Background 0 0.2 0.4 0.6 0.8 1 Distance from the center of the drop Rain Drops are Brighter than Background Brightness is Independent of Background

  10. The World in a Drop A Drop World in a Drop Perspective Views of the World

  11. Rain and Background Intensities High Shutter Speed ( Exposure Time = 1/1000 s ) Normal Shutter Speed ( Exposure Time = 1/30 s )

  12. Pixel Frame Motion Blur Model for Rain Drops Irradiance Stationary Rain Drop Background Pixel Background Time Exposure Time Time a drop stays over a pixel : Change in Intensity : Rain produces a “Delta Signal” in Time

  13. Motion Blurred Intensities of Rain Streaks Irradiance Intensity at a Pixel on a Rain Streak : Stationary Rain Drop Intensity of the Background : Background Time Exposure Time Change in Intensity due to a Rain Streak : Constants Constants Change in Intensity is Linearly related to Background

  14. Dynamics of Rain Background Pin Hole Frames Time Y-axis X-axis Discrete Space Time Volume Continuous Space Time Domain Rain Produces Strong Directional Correlations in Videos

  15. Rain Detection and Removal Algorithm Linear Photometric Constraint Select Delta in time Estimated Rain Pixels Candidate Rain Pixels Original Video Neighboring Frames Compute direction and strength Dynamic Constraint Rain Segmented Correlation Map Estimated Rain Field

  16. Results: A clip from Magnolia Original Clip Courtesy © New Line Productions, Inc. Rain Detection

  17. Results: A clip from Magnolia Original Video Derained Video Rain Component

  18. Results: Rain vs Ripples Original Video Rain Segmentation

  19. Results: Rain vs Ripples Original Video Derained Video Rain Component

  20. Results: Person Walking in Rain Original Video Rain Detection

  21. Results: Person Walking in Rain Derained Video Original Video Rain Component

  22. Summary • Visual Appearance of Rain + = Dynamics Rain Photometry • Rain Detection and Removal • Future Work: Snow Dynamics Photometry

More Related