Technion - Institute of Technology faculty of electrical engineering

1. Technion - Institute of Technology faculty of electrical engineering computer vision and image science lab Restoring Distorted Images of Objects Submerged Under water reuven shefer & maor malchi supervisor: arie shenar 5.2.2001

2. Problem Description: • Images submerged underwater appear distorted because of the presence of surface waves. • The distortion is categorized by 3 operations: shrinking, stretching and shifting of objects. Assumptions: • The exact degree of the distortion isn’t known when reconstructing the images. • The wave distortion doesn’t cause raptures in the objects of the image.

3. Example of The Distortion Type: • The square was distorted by a shrinking distortion • the ellipse was distorted by a shifting distortion

4. Image Example:

5. Quantitative characterization of the distortion: • The degree of the distortion could be found by comparing the position or the size of the photographed object to its size or position in the image without waves. • The nature of the distortion could be determined by testing the position of each pixel of the object, in shifting distortion the pixels translate together, in shrinking the pixels translate from the outer borders of the object inward.

6. The project concept The image without waves is unavailable. • Working with a number of distorted images taken in different instances of time. Working with objects of the image is difficult. • Instead of working with objects, the image is divided into a grid of blocks. • For each block in image A, the algorithm find a block in image B, which is the best match according to a preselected comparing operator .

7. The project concept (continued) How can we use the estimation of the distortion to restore the image? • Given the distortion estimation between two images, the images are “distorted” toward one another to form the reconstructed image.

8. Block Matching Between Two Images: C D E • Block D is the one we trying to match, Block E is the best match in image B, the search environment is market by the letter C. • dx ,dy are kept in a displacement matrix.

9. Interpolation: • interpolation of the translation matrix is needed because the translation matrix doesn’t describes the movement of individual pixels. Steps for restoring 2 images (forward projection): • Calculate translation matrix • Interpolate translation matrix • translate each pixel according to the translation matrix into the restored image. • the value of Pixels that fall in the same pixel of the restored image will be averaged.

10. Problem: the translation matrix after interpolation contains distances that aren’t an integer number of pixels. • Solution: the value of the pixel will be divided between the four pixels of the restored image that it was mapped to, according to area ratio. 1) the mapped pixel is outlined by blue frame. 2) pixel number 1 get 1/9 of the mapped pixel value. 3) pixel number 2 gets 2/9 of the mapped pixel value 4) pixel number 3 gets 2/9 of the mapped pixel value 5) pixel number 4 gets 4/9 of the mapped pixel value.

11. Forward Projection Summery • A simple and fast method. • “over mapping” and “under mapping” , seen in the restored image as artifacts (black or white lines). The solution is Backward Projection

12. Steps of Backward Projection Algorithm: • Calculate translation matrix. • Calculate reversed translation matrix. • Interpolate the reversed translation matrix. • Pixel values are collected from the distorted image, according to the translation matrix value. • This method prevents “over mapping” or “under mapping”.

13. Image PreProcessing Block Builder Translation Matrix Calculator Interpolator Reconstruction Unit Image PostProcessing Backward Projection Algorithm:

14. Merging Tree of 4 Distorted Images: Restored image Intermediate image 1 Intermediate image 2 Distorted image 1 Distorted image 2 Distorted image 3 Distorted image 4

15. Test Images:

16. Difference images:

17. Resorted Image:

18. Resorted Image Analysis:

19. Average Between 4 Distorted images

20. Averaged Image analysis.

21. Qualitative Measurements of The Restoration: • correlation. • mean absulute value • difference image of the restored image and image without waves • Observation comparison of object in the restored image against the same object in the image without waves (with several observers).

22. System Parameters: Parameters which improve system performance: • Block type and size. • Filtering distorted images. • Input image merging sequence. Parameters which improve system performance in case of incorrect block type or size: • Finer sampling grid. • Translation vectors correction algorithm.

23. System Performance: • System can fix shifting distortion, and slow varying shrinking or stretching distortion. • 12% improvement in correlation measurements. • 25%Decrease in The average absolute difference. • Fine details like letters can be restored (if they fall inside a block).

24. System Faults: • The system uses constant parameters, so the restoration for some areas of the image may not be optimal. • The system is based on block shifting, so it cannot handle strong stretching. • There is a large amount of blurring.

25. Synthesized Distorted Images:

26. Restoring Synthesized Distorted Images: Image without distortion restored image

27. Correlation and Average Absolute Difference Values:

28. Suggestions For A Future Project: • Varying block size or type for different areas of the image. • Calculate translation matrix for blocks that their center is on an edge point. • Use gaussian weight function on the search environment.