1 / 28

SKETCH-BASED USER INTERFACE STUDY

SKETCH-BASED USER INTERFACE STUDY. Presented By Jin Xiangyu Department of Computer Science and Technology Nanjing University  June 2002. PART I: INTRODUCTION. The rise of the research issue of Human-Computer Interaction (HCI). Computer-Oriented. Human-Oriented.

amy-oliver
Download Presentation

SKETCH-BASED USER INTERFACE STUDY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SKETCH-BASED USER INTERFACE STUDY Presented ByJin Xiangyu Department of Computer Science and TechnologyNanjing University June 2002

  2. PART I: INTRODUCTION

  3. The rise of the research issue of Human-Computer Interaction (HCI) Computer-Oriented Human-Oriented 1.1. A Revolution is Undertaking Computers are becoming more and more powerful and easily available today Human and computer, which one should be the center of computer assisted tasks? The idea of this revolution is to bend computers to people’s way of interacting, not the other way around (Landay 2001)

  4. 1.2. Why Sketch-based User Interface? To write down designer’s improvisatory ideas by diagrams is very important for creative tasks. Traditional menu/toolbar button-based user interface Demo (1) Inefficient A three-step process. (2) Unnatural Leaving a sketch uninterrupted, or at least in its rough state, is key to preserving this fluidity (Hearst 1998). (3) Unsuitable for handled devices No area to accommodate so many stencils and buttons. The Solution is “By Sketch”. Sketching with a pen is a mode of informal, perceptual interaction that has been shown to be especially valuable for creative design tasks (Gross 1996) .

  5. Three application level Level Symbol Set Drawing Approach Example 1 Strictly defined Generally agreed among users Handwriting recognition 2 Not very strictly defined Stroke-number and stroke order free Sketchy symbol recognition 3 Undefined Totally free Sketch-based image retrieval 1.3. Research focus: What Kind of Sketch-based User Interface We are Interested in? On-line VS Off-line Our research focus

  6. 1.4. Three Designing Principles: Humanistic, Intelligent, Individualized How to make the UI to be humanistic? An graphics inputting user scenario is proposed, which employs an interactive sketching-recognition-rectification process in one-fluent-step. How to make the UI to be intelligent? On-line graphics recognition is employed to predict user’s original intention. How to make the UI to be individualized? SVM-based incremental learning is employed to adapt different user in shape classification. These three characteristics are harmoniously combined in our prototype system —Smart Sketchpad.

  7. PART II: HIGHLIGHTS

  8. Step 1: Sketching Recognize and Regularize The suggested candidate objects Step 2: Sketching Recognize and Regularize Step 3: Clicking on the intended object and replace the strokes with the very object with proper parameters 2.1. Critical Technique 1: User Scenario for Graphics Inputting Employing one interactive, fluent sketching-recognition-rectification process instead of three split ones.

  9. Strokes Primitive Shape Primitive Shape Composite Graphic Object Primitive Shape Classification and Regularization Composite Shape Recognition 2.2. Critical Technique 2: On-line Graphics Recognition Sketching (user) : Decompose Composite Graphic Object Recognition (computer) : Assemble Strokes

  10. The User Sketchy Shape The User Intended Shape Preprocessing Shape Classification Shape Fitting Shape Regularization The input stroke Quadrangle The fitted shape The regularized shape 2.2.1. Primitive Shape Classification and Regularization By Vertex Combination Primitive Shape Classification & Regularization Strokes Primitive Shapes

  11. User2 0.35 User1 0.75 The optimal thresholds are different Experimental Results Shape Classification Precision for 1367 samples

  12. Inner-Shape Regularization The results of primitive shape classification and regularization Experimental Results Shape Regularization Results

  13. In order to suggest the user in an early stage, the system should recognize graphic object in an incomplete form. 2.2.2. Composite Graphic Object Recognition A “Partial”“Structural” Similarity Assessment Strategy is Proposed

  14. The similarity assessment strategy should not only invariant to shifting, rotation, mirroring, but also should invariant to inner distortion. 2.2.2. Composite Graphic Object Recognition A “Partial”“Structural” Similarity Assessment Strategy is Proposed

  15. The Source Object The Candidate Object L1 P1 Graphic Primitive Extraction L2 L3 P2 P5 P3 Line-segments, arc-segments, and ellipses/circles L4 P4 Spatial Relation Graph (SRGs) The Proposed Approach The computational complicity is Pnm. Conditioned Partial Permutation Algorithm

  16. The Original Object Adding Noises Eliminating some parts The Generated Query Performance Evaluation Query Generating by adding noises and eliminating some parts. When the user draws 80% of his/her intended object (for users may miss some parts of the object inadvertently) with 10% distortion (this is similar to noises in real user drawing situation), R6 is nearly 90%(averagely of 304 graphic objects). Experimental results show that our approach can achieve good performance with noises for incomplete objects, and our approach is also invariant to shifting, rotation, mirroring, and inner distortions.

  17. An ambiguous case Question: A triangle or a quadrangle? 2.3. Critical Technique 3: User Adaptation User adaptation is a classical problem in user interface study. Many pattern recognition problems are user specific, for users’ handwritings, drawing styles, and accents are different. Rule-based feedback may yield “conflict” results due to its intrinsic deficiency, which may lose its general performance when it adapts to a specific user further. SVM-incremental learning are introduced into the user adaptation problem of shape classification.

  18. 2.3.1. Questions • Four questions need to be solved: • Whether SVM-based Incremental learning can overcome “conflicts”? • What is the advantage of Incremental leaning compared with repetitive learning? • Which one is better, Syed’s or Xiao’s? • Which structure is better, one-against-one and one-against-all? 2.3.2. Experiments Experimental Environments: Feature Extraction (20-dimensional vector) by turning function Virtual Sample Generation (with 40 samples each) 40 incremental training sets and two test sets are created Training time, open-test precision, closed-test precision are tested for different algorithms and structures.

  19. 2.3.3. Answers Theoretical analysis and experimental results both show (1) SVM-based incremental leaning can overcome “conflict” (2) Incremental learning is much faster than repetitive learning without loss of precision (3) Syed’s algorithm is better than Xiao’s (4) One-against-one structure is much faster than one-against-all in our environments

  20. PART III: THE SMART SKETCHPAD SYSTEM

  21. 3.1. System architecture of Smart Sketchpad

  22. The Sketching Area The just inputted shape are recognized and regularized Part of the intended object are sketched The candidate shape are shown Candidate object name and their similarities are shown Candidate object list 3.2. The Sketch-based User Interface of Smart Sketchpad Inputting two graphic objects and then delete them Demo

  23. 3.3. UI Evaluation 10 different subjects are required to draw the following two diagrams with traditional UI and the sketch-based UI. There are 304 objects listed in 26 stencils (with 12 each) for traditional UI. There are 6 objects can be shown in the Smart Toolbox for sketch-based UI. Diagram 1 Diagram 2 A demo of inputting Diagram 2 by sketch

  24. Drawing Time for Different Sketches Under Different UIs Averagely, the sketch-based UI is 22.4% and 42.9% more efficient than the traditional toolbar button-based UI for sketch1 and sketch2, respectively. The ultimate comments of all users unanimous agree that they’d like to choose the sketch-based UI instead of the traditional one.

  25. PART IV: SUMMAY

  26. Algorithm Level (1) Agglomerate Point Filtering, Vertex Combination, Shape Fitting, Shape Regularization (2) Conditioned Partial Permutation (3) Comparison of SVM-based Incremental Learning Algorithms and Structures Solution Level (1) Interactive graphics inputting user scenario (2) On-line graphics recognition: primitive shape classification and regularization; composite shape recognition. (3) SVM-incremental learning for user adaptation in shape classification problem. System Level (1) A prototype system for conceptual/schematic designing tasks is implemented. (2) User evaluation between the traditional and sketch-based UIs is performed.

  27. Future Works (1) How to perform stroke segmentation? (2) How to cut down the computational cost and improve the recognition precision? (3) How to make the system learn aggressively?

  28. THANKS

More Related