1 / 65

H A N D T A L K

H A N D T A L K. An Interactive Web-Based Sign Language Modeler. The Proponents. Natalie Rubnan Que Maria Melissa Mejia Marjorie Joy Pantig Jackson Velasco. Overview of Research. HandTalk – translation of sign language coupled with an instant messenger

kaseem-bond
Download Presentation

H A N D T A L K

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. H A N D T A L K An Interactive Web-Based Sign Language Modeler

  2. The Proponents . . . • Natalie Rubnan Que • Maria Melissa Mejia • Marjorie Joy Pantig • Jackson Velasco

  3. Overview of Research • HandTalk – translation of sign language coupled with an instant messenger • Motion Capture – realistic hand movement • ASL (American Sign Language) – widely used language • Limited to fingerspelling and goes back to open palm position

  4. Overview of Research • Research . . . • 3D Graphics on the Web • Animation Techniques • Web Application Implementation

  5. Quick Demo Here?

  6. Overview of Research • Research Objectives Met • Study basic sign language • Review sign language related papers , software, theses • Study formulas, algorithms about hand modeling • Study joints of the hand • Study use of motion capture data • Investigate animation techniques • Study web application issues • Apply HCI concepts in system design

  7. Overview of Research • System Objectives Met • Display a 3D hand model on a web page in WWW. • Provide function to choose which hand • Provide functions for modeling transformations on the model • Provide a function that would accept user text input • Provide a means for the user to choose whom the receiver of the text input is • Provide a function that would parse input and divide into token stream • Provide a function that would accept token stream and retrieve motion data

  8. Overview of Research • System Objectives Met • Provide a function that would enable 3D hand model to perform fingerspelling motion of user input • Provide an option to set speed of translation • Provide a means for the user to identify himself in system • Provide a means for the user to keep track of his/her contacts list • Provide a means for the user to keep track of the messages he/she received

  9. Overview of Research • General Objective: “To research on the design and implementation of an interactive web-based sign language modeler.”

  10. Overview of Proposed System • Three Major Modules: • Hand Modeler • Sign Modeler • Sign Language Messenger

  11. Overview of Proposed System • Overall Structure of the System

  12. Maybe we could make a Flash movie here that shows how system works during operation

  13. Proposed System…Hand Modeler • Hand Modeler – hand model generation • Problem: Which 3D graphics tool? • Choices: • VRML • Microsoft .NET Technology • Macromedia Director Shockwave

  14. Proposed System…Hand Modeler • Chosen Solution . . . • Macromedia Director Shockwave • Wide range of multimedia applications • VRML slower and limited • Revolution 3D, not for the Web

  15. Proposed System…Hand Modeler • Shockwave Control • Sprites • Scripts • State flags

  16. Proposed System…Hand Modeler • Major Functions: • Select Hand Model • Rotation • Camera View • Adjust Lighting • Reset All

  17. Hand Modeler Functions • Select Hand Model • Right, Left or Both Hands • Right Hand values negated for Left Hand • Hierarchy of Joints

  18. Hand Modeler Functions • Select Hand Model • Problem # 1: Speed Issues • Solution: Trade between realism and speed

  19. Hand Modeler Functions • Select Hand Model • Problem # 2: Default Orientation of Hand • Solution: Adjustment of Camera

  20. Hand Modeler Functions • Rotation • Rotate about X, Y, Z axis • Root Joint Only • 10 degrees • Rotation of Model Itself, not Camera

  21. Hand Modeler Functions • Rotation • Problem # 1: Inconsistent hand orientations due to a combination of rotations • Solution: A list instead of state flags

  22. Hand Modeler Functions • Camera View • Translation and Scaling • State flags • Camera instead of model

  23. Hand Modeler Functions • Adjust Lighting • Brightness and Contrast • Contrast disabled

  24. Hand Modeler Functions • Adjust Lighting • Problem: Default Director light • Solution: Lighting from scripts

  25. Hand Modeler Functions • Adjust Delay • Delay between frames • State flag

  26. Hand Modeler Animation • Frame-by-frame • Data for specific frame applied • Adjust from user controls

  27. Proposed System…Sign Language Modeler • Sign Language Modeler – retrieve data and pass to Hand Modeler • Three languages: • Java • JavaScript • Shockwave Lingo • Data on client side – minimum network delay

  28. Proposed System…Sign Language Modeler • Sign Language Modeler • Problem: Slow retrieval of data • Solution: Streaming of data

  29. Sign Language Modeler Functions • Process Text • Read user input • JavaScript for speed and Evalscript • Validation in JavaScript, not Java

  30. Sign Language Modeler Functions • Process Motion Data • Token = parsing of motion data in BVH file • Digitally signed Java applet • Dependent on structure of BVH file • String tokenizer: only offset values (OFFSET part), number of frames, motion data (FRAMES part) • Comma delimiter (,)

  31. Sign Language Modeler Functions • Process Motion Data • variable data: faster tokenizing, shorter strings to append • Data sent back as string of form: <character translated> <offsets> <number of frames> <motion data>

  32. Sign Language Modeler Functions • Process Motion Data • Problem # 1: Different natures of security certificates in Netscape and IE • Solution: Two implementations • Problem # 1: No permission • Solution: Netscape asks again, IE need to restart

  33. Sign Language Modeler Functions • Map Joint Angle Data • Shockwave Lingo • Offset and motion data to joints • Data from Process Motion Data (Java) through JavaScript function Evalscript • Maps specific data in every time frame

  34. Sign Language Modeler Functions • Map Joint Angle Data • variable MocapData: data from Evalscript • Values retrieved from MocapData based on form of string • variable PassData: currentmotiondata to be mapped

  35. Sign Language Modeler Functions • Map Joint Angle Data • Problem: Shared data conflict due to concurrency • Choices: • Binary Semaphores • Semaphores • Wait-Signal Mechanisms • Solution: Binary Semaphores – simple, yet adequate and effective

  36. Proposed System…Sign Language Messenger • Instant messaging • Separation of major functions instead of centralization – no bottlenecks

  37. Sign Language Messenger…The Server Program • All the users, all the messages • instant_messenging class: client >> web server >> server program

  38. Sign Language Messenger…The Server Program • Keeping Track of Users… • client_thread group: dynamic storage, no waste of time in file access • Different functions for interactivity (threads) client_threadgroup.write_message_to(x, sender + "¿" + recipient + "¿" + msg);

  39. Sign Language Messenger…The Server Program • Keeping Track of Users… • Problem: Information transfer between threads • Choices: • Arrays • Vector Lists • Solution: Vectors: for distributed computing

  40. Sign Language Messenger…The Server Program • Enabling User Connection • server_temp_thread: thread-embedded socket to a client • Server alert from client • Necessary functions first int x = client_threadgroup.search_for_username(s); CLIENT: socket = new Socket("server ip", port); SERVER: server_temp_thread t = new server_temp_thread(new_socket); t.start();

  41. Sign Language Messenger…The Server Program • Enabling User Connection • Problem: Checking if logged before acceptance as user • Solution: client_thread_group static, so accessible

  42. Sign Language Messenger Functions • Create New Account • add_db: add to db • check_existing • Prepared statements instead of one-by one insertion: faster

  43. Sign Language Messenger Functions • Login • Database instead of files, faster • Problem: Status of login • Solution: • lok: login ok • lusok: password incorrect • lnotok: login not ok

  44. Sign Language Messenger Functions • Details • Change or retrieve password • Database instead of files, faster

  45. Sign Language Messenger Functions • Contacts List • contacts class • File instead of database, too much space

  46. Sign Language Messenger Functions • Message: Read Messages • Thread • Format of message: <sender> <recipient> <message> • “¿”: delimiter

  47. Sign Language Messenger Functions • Message: InTray • Files instead of database: avoid redundancy

  48. Sign Language Messenger Functions • Message: Send Messages • User message restructured into defined format

  49. Sign Language Messenger Functions • Log-out • Close session

  50. Sign Language Messenger Misc. Design Issues • Java Server Pages vs. Java Servlets • JSP over ASP • JSP same functionality as servlet • Presentation from logic • Mixed HTML and code? Java Beans.

More Related