1 / 13

Sign Language Recognition

Sign Language Recognition Overview Average person typing speed Composing: 19 words per minute Transcribing: 33 words per minute Sign speaker Full sign language: 200—220 wpm Spelling out: est. 50-55 wpm Faster by 1.5x to 3x Purpose and Scope Native signers can input faster Benefits:

jaden
Download Presentation

Sign Language Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Sign Language Recognition

  2. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Overview • Average person typing speed • Composing: 19 words per minute • Transcribing: 33 words per minute • Sign speaker • Full sign language: 200—220 wpm • Spelling out: est. 50-55 wpm • Faster by 1.5x to 3x

  3. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Purpose and Scope • Native signers can input faster • Benefits: • Hearing & speaking disabled • Sign interpreters • Just letters & numbers for now

  4. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Research • Related projects • Using mechanical gloves, colored gloves • Tracking body parts • Neural network-based application • Still images: 92% accuracy • Motion: less than 50% accuracy • Feature vector-based application • Also about 90% accuracy on stills • No motion tests

  5. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 More Research • Image techniques: • Edge detection (Robert’s Cross)‏ • Line detection (Hough transform)‏ • Line interpretation methods • Chaining groups of lines • Macro-scale templates

  6. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Program Architecture • Webcam interface • Edge detection • Line detection • Line interpretation • Hand attribute matching

  7. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Testing Model • Human interaction necessary • General testing model: bhood@bulusan:~/syslab-tech$ \ > ./main images/hand.pgm in.pgm [DEBUG] Edge detect time: 486 ms [DEBUG] Wrote image file to `in.pgm’ Errors:0Warnings:0

  8. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Code sample • Robert’s Cross edge detection: int row,col; // loop through the image for(row=0; row<rows; row++) { for(col=0; col<cols; col++) { // avoid the problems from trying to calculate on an edge if(row==0 || row==rows-1 || col==0 || col==cols-1) { image2[row][col]=0; // so set value to 0 } else { int left = image1[row][col-1]; // some variables int right = image1[row][col+1]; // to make the final int top = image1[row-1][col]; // equation seem a int bottom = image1[row+1][col]; // little bit neater image2[row][col]=(int)(sqrt(pow(left – right , 2) + pow(top - bottom, 2))); } } }

  9. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Edge Detection Results Original image (800 x 703)‏ • Results: • Shows the important edges but little else • Robert’s Cross method the optimal balance Final image (800 x 703)‏

  10. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Cropping Results • Remove useless rows with no features • Very large optimization • Area difference • Input/output Original (800 x 703)‏ Result (633 x 645)‏

  11. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Line detection • Mostly complete • Still some bugs to iron out • Shows nonexistent lines occasionally (see image!)‏

  12. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 Line Interpretation • Chaining groups of lines • Templates • Generation • Template-based comparison • Hand matching

  13. Byron Hood | version 0.3 Computer Systems Lab Project 2007-2008 The Mysterious Future • Finish line interpretation • Building method for interpreting lines • Working on template-based recognition • Write XML template files • Finish camera-computer interaction • Device control must be precise, picky

More Related