1 / 28

EEC-492/592 Kinect Application Development

Understand and apply image processing methods using Emgu CV for Kinect applications. Learn about image filtering, edge detection, and shape detection. Dive into the realm of convolution, filters, and image blurring.

kgalvan
Download Presentation

EEC-492/592 Kinect Application Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EEC-492/592Kinect Application Development Lecture 18 Wenbing Zhao wenbing@ieee.org

  2. Outline Image processing with Emgu CV Image Filtering Edge detection Shape detection

  3. Image Processing vs. Computer Vision • Image processing mainly deals with getting different representations of images by transforming them in various ways • Computer vision is concerned with extracting information from images so that one can make decisions

  4. Image Filters • Image filter: a function that takes the original pixel and gives an output that is proportional in some way to the info contained in the original pixel • Image filter is 2 dimensional matrix, referred to as a kernel • Images are 2 dimensional • A filter matrix is a discretized version of a filter function • A filter for filtering out vertical edges: 0 0 0 0 0 0 0 0 0 0 -1 -2 6 -2 -1 0 0 0 0 0 0 0 0 0 0 // Filter kernel for filtering out vertical edges float vertical_fk[5][5] = {{0,0,0,0,0}, {0,0,0,0,0}, {-1,-2,6,-2,-1}, {0,0,0,0,0}, {0,0,0,0,0}}; // Filter kernel for filtering out horizontal edges float horizontal_fk[5][5] = {{0,0,-1,0,0}, {0,0,-2,0,0}, {0,0,6,0,0}, {0,0,-2,0,0}, {0,0,-1,0,0}};

  5. Applying Filters (Convolution) • Element-wise multiplication between elements of the kernel and pixels of the image • A function is used to calculate a single number using the result of all these element-wise multiplications • This function can be sum, average, minimum, maximum, or something very complicated • The value thus calculated is known as the “response” of the image to the filter at that iteration • The pixel falling below the central element of the kernel assumes the value of the response • The kernel is shifted to the right and if necessary down

  6. Convolving Filter: Example • From: http://www.roborealm.com/help/Convolution.php • A grayscale image (10x10)

  7. Convolving Filter: Example • A filter for line detection • The row=2, col=2 pixel and its neighborhood • Multiply the filter values with the image block: work with each pixel and its 3x3 neighborhood

  8. Convolving Filter: Example (-34)+(-22)+(-77)+(-50)+(1200)+(-77)+(-93)+(0)+(-77) = 770 • Sum of all values: • Divide by the divisor and add the bias • Divisor and bias might be needed to keep pixel value within 0 to 255 • Typically divisor=1, bias=0: final result remains to be 770 • If new pixel value is > 255, set it to 255 • If new pixel value is < 0, set it to 0 • Hence, new pixel value for row=2, col=2 is 255

  9. Convolving Filter: Example • Continue with all other 3x3 blocks using original values • Next block

  10. An Implementation of 2D Filtering private Image<Bgr, Byte> filterImage(Image<Bgr, Byte> image, double[,] filter, double factor, double bias) { Image<Bgr, Byte> filtered = new Image<Bgr, Byte>(image.Width, image.Height); int w = image.Width; int h = image.Height; int filterWidth = filter.GetLength(0); int filterHeight = filter.GetLength(1); byte[, ,] data = image.Data; byte[, ,] filteredData = filtered.Data; for (int y = 0; y < w; ++y) { for (int x = 0; x < h; ++x) { double red = 0.0, green = 0.0, blue = 0.0; for (int filterY = 0; filterY < filterWidth; filterY++) { for (int filterX = 0; filterX < filterHeight; filterX++) {

  11. An Implementation of 2D Filtering { int imageY = (y - filterWidth / 2 + filterX + w) % w; int imageX = (x - filterHeight / 2 + filterY + h) % h; red += data[imageX, imageY, 2] * filter[filterX, filterY]; green += data[imageX, imageY, 1] * filter[filterX, filterY]; blue += data[imageX, imageY, 0] * filter[filterX, filterY]; } byte r = (byte)Math.Min(Math.Max((int)(factor * red + bias), 0), 255); byte g = (byte)Math.Min(Math.Max((int)(factor * green + bias), 0), 255); byte b = (byte)Math.Min(Math.Max((int)(factor * blue + bias), 0), 255); filteredData[x, y, 0] = b; filteredData[x, y, 1] = g; filteredData[x, y, 2] = r; } } } return filtered; }

  12. Filtering Results

  13. Blurring Images • Blurring an image is the first step to reducing the size of images without changing their appearance too much • An image can be thought of as having various “frequency components” along both of its axes’ directions • Edges have high frequencies, whereas slowly changing intensity values have low frequencies • Vertical edge creates high frequency components along the horizontal axis of the image and vice versa. • Finely textured regions have high frequencies (pixel intensity values in it change considerably in short pixel distances)

  14. Blurring Images • Reduce the size of an image: remove high-frequency components from it • Smooth out those high-magnitude short-interval changes • Image Intensity: Sum of the values at a pixel • RGB color image • Three channels: R, G, B • Each color channel’s intensity is the value of the color • Low intensity: dark • High intensity: whiter

  15. Blurring Images • Blurring can be accomplished by replacing each pixel in the image with some sort of average of the pixels in a region around it • To do this efficiently, the region is kept rectangular and symmetric around the pixel, and the image is convolved with a “normalized” kernel • Normalized because we want the average, not the sum

  16. Blurring Images: Kernels 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 • Box kernel • Sum: 25 • Gaussian kernel • Sum: 256 1 4 6 4 1 4 16 24 16 4 6 24 36 24 6 4 16 24 16 4 1 4 6 4 1

  17. Blurring Images: Results • Reuse the same implementation as that for image filtering except we must normalize the calculation • Divide by the sum of the kernel used

  18. Edge Detection • Edges: points in the image where the gradient of the image is quite high • Gradient: change in the value of pixel intensity • The gradient of the image is calculated by calculating gradient in X and Y directions and then combining them using Pythagoras’ theorem (a2 + b2 = c2) • The angle of gradient can be calculated by taking the arctangent of the ratio of gradients in Y and X directions, respectively • Kernels for calculating gradients along x and y direction • X direction: y direction: -3 -10 -3 0 0 0 3 10 3 -3 0 3 -10 0 10 -3 0 3

  19. Canny Edges • The Canny algorithm uses some postprocessing to clean the edge output and gives thin, sharp edges • Canny algorithm • Remove edge noise by convolving the image with a normalized Gaussian kernel • Computer x and y gradients by using kernels: • Find overall gradient and gradient angle. Angle is rounded off to 0, 45, 90 and 135 degrees • Non-maximum suppression: A pixel is considered to be on an edge only if its gradient magnitude is larger than that at its neighboring pixels in the gradient direction • Hysteresis thresholding: use two thresholds • A pixel is accepted as an edge if its gradient is higher than the upper threshold, rejected if smaller than lower threshold • For in-between pixels, it is accepted as an edge only if it is connected to a pixel that is an edge -1 -2 -1 0 0 0 1 2 1 -1 0 1 -2 0 2 -1 0 1

  20. Canny Edge Detector API public Image<TColor, TDepth> Canny(TColor thresh, TColor threshLinking); • Defined for Image<> class • Parameters: • thresh: The threshhold to find initial segments of strong edges • threshLinking: The threshold used for edge Linking • Returns: The edges found by the Canny edge detector

  21. Corners Detection • A corner can be defined as the intersection of two edges • A corner can also be defined as a point for which there are two dominant and different edge directions in a local neighborhood of the point

  22. Building ImageFiltering App • Create a new C# WPF project named “ImageFiltering” • Add references, copy opencv dlls to path, configure project property for x64 bit target • Design GUI • Two image controls, 2 labels, a textbox, 6 buttons • Adding code

  23. Building ImageFiltering App

  24. Building ImageFiltering App • Import name spaces • using System.Diagnostics; • using System.Drawing; • using System.Text; • using Emgu.CV; • using Emgu.CV.Features2D; • using Emgu.CV.Structure; • using Emgu.Util; • using System.Runtime.InteropServices; // for DllImport

  25. Building ImageFiltering App • Core method for convolving (portion shown earlier) • Takes an Image<> and a filter (kernel) with parameters • For blurring, filter must be normalized • Returns the filtered image private Image<Bgr, Byte> filterImage(Image<Bgr, Byte> image, double[,] filter, double factor, double bias, bool normalize=false) if (normalize) { // calculate the sum of all elements in the kernel double sum = 0; for (int i = 0; i < filterWidth; i++) { for (int j = 0; j < filterHeight; j++) { sum += filter[i, j]; } } // need to normalize the filter kernel for (int i = 0; i < filterWidth; i++) { for (int j = 0; j < filterHeight; j++) { filter[i, j] = filter[i, j] / sum; } } }

  26. Building ImageFiltering App • Filters used: vertical, horizontal, blur, sharper • Always use factor=1.0 and bias=0 // Filter kernel for detecting vertical edges double[,] verticalFilter = new double[,] { { 0, 0, 0, 0, 0 }, { 0, 0, 0, 0, 0 }, { -1, -2, 6, -2, -1 }, { 0, 0, 0, 0, 0 }, { 0, 0, 0, 0, 0 } }; // Filter kernel for detecting horizontal edges double[,] filter = new double[,] { { 0, 0, -1, 0, 0 }, { 0, 0, -2, 0, 0 }, { 0, 0, 6, 0, 0 }, { 0, 0, -2, 0, 0 }, { 0, 0, -1, 0, 0 } }; // Filter kernel for blurring (Gaussian kernel) double[,] filter = new double[,] { { 1, 4, 6, 4, 1 }, { 4, 16, 24, 16, 4 }, { 6, 24, 36, 24, 6 }, { 4, 16, 24, 16, 4 }, { 1, 4, 6, 4, 1 } }; // sharper filter double[,] filter = new double[,] { { -1, -1, -1 }, { -1, 9, -1 }, { -1, -1, -1 } };

  27. Building ImageFiltering App • Edge detection code //Convert the image to grayscale and filter out the noise Image<Gray, Byte> gray = img.Convert<Gray,Byte>(). PyrDown().PyrUp(); // to remove noise Gray cannyThreshold = new Gray(180); Gray cannyThresholdLinking = new Gray(120); Image<Gray, Byte> cannyEdges = gray.Canny(cannyThreshold, cannyThresholdLinking); image2.Source = ToBitmapSource(cannyEdges);

  28. Building ImageFiltering App // Create OpenFileDialog Microsoft.Win32.OpenFileDialog dlg = new Microsoft.Win32.OpenFileDialog(); // Display OpenFileDialog by calling ShowDialog method Nullable<bool> result = dlg.ShowDialog(); // Get the selected file name and display in a TextBox if (result == true) { Image<Bgr, Byte> newimg; try { newimg = new Image<Bgr, byte>(dlg.FileName); this.img = newimg; image1.Source = ToBitmapSource(this.img); } catch { MessageBox.Show("Invalide file format"); return; } } • Browse and load a new image

More Related