Designing a Handwriting Recognition Based Writing Environment

1. Designing a Handwriting Recognition Based Writing Environment J C Read, S J MacFarlane, C Casey Department of Computing, University of Central Lancashire, Preston, UK

2. Introduction • Background Information • Description of the Observational Study • Findings from the study • General Findings • Findings relating to the Handwriting Recognition • Satisfaction measuring • Informing the design • Description of the Prototype • Conclusions JCRead@uclan.ac.uk

3. Who? Why? What? Where? How? • Janet C Read, lecturer and mother! • Elodie, PhD study • Speech, handwriting: - Human Language Technology, Free text not command. • Lancs..UK; white rural primary – age 7 – 9 • Research, Observations, Usability studies JCRead@uclan.ac.uk

4. Previous Work • QWERTY keyboard difficult (WES2000) • HLT attractive to children, HR feasible (HCI2000, HCI2001) • Measuring Fun (CandF2000, CandF2001) • Participatory design (IDC2002) • Errors in HR interfaces (NordiCHI2002) JCRead@uclan.ac.uk

5. The Observational Study • Children aged 7 and 8 • Normal classroom activities • In twos • Laptop (HR), Desktop (QWERTY), Desk (Pencil) • Different writing tasks • Difficulties, Errors, Corrections JCRead@uclan.ac.uk

6. Pen and Paper Errors • Errors made – missing words, spelling, letters written backwards • Error prevention – asked, avoided, looked • Error discovery – reading back, self, teacher or another child • Error repair – rub out, scribble out, cross out, overwrite, re-write, squeeze in , change JCRead@uclan.ac.uk

7. QWERTY - Errors • Errors made – missing words, spelling, hit wrong key • Error prevention – asked, avoided, looked • Error discovery – reading back, self, teacher or another child, wiggly lines! • Error repair – position and rub out, rub back to, rub all, retype, change JCRead@uclan.ac.uk

8. Handwriting Recognition - Overview • Hardware – Graphics tablet and pen • Software – Recognition software • Fuzzy computing • Disobedient – ambiguous • Character or word based • On line – ‘t’ stroke problems • Demonstration JCRead@uclan.ac.uk

9. Demonstration of handwriting JCRead@uclan.ac.uk

10. Handwriting Recognition – Errors (1) • Child • Errors made – miss words, spellings, letters backwards, pen up • Computer • Errors made – Bad recognition, hardware • Child • Error prevention – ask, avoid, look • Computer • Error prevention – spell checker (not used) JCRead@uclan.ac.uk

11. Handwriting Recognition – Errors (2) • Child • Error discovery • Before Recognition – reading back, teacher, other child • After Recognition – as above + wiggly lines! • Error repair • Before recognition – scribble out, overwrite, insert letter • After Recognition – rub back to, rub all, rewrite all or some, use QWERTY JCRead@uclan.ac.uk

12. Satisfaction Measuring • Errors do not imply dissatisfaction • WHY? • Sticky – addictive vs. nothing better • Funny – humour with recognition – easy to use JCRead@uclan.ac.uk

13. Designing a Prototype - method • Users • Children, environment, characteristics, mental models • Tasks • Goal oriented – hierarchy • System • States – dangerous states • Interface • UI design guidelines JCRead@uclan.ac.uk

14. Child • Classroom based – standard equipment, needs to be easy to use, robust, minimal help needed • Children – varied pen control, different levels of expertise with technology, different reading skills, poor or very good letter formation • Mental model – see tablet as paper – want to scribble out and insert missed words JCRead@uclan.ac.uk

15. Child writing JCRead@uclan.ac.uk

16. User Goal • To produce good written work • Planning • Translation • Reviewing and Editing (Hayes and Flower) JCRead@uclan.ac.uk

17. Supporting the writer (1) • Ideas – pop up in clouds, can have many, child can re-order them and can put them away, use handwriting that is not recognised • Translation – training supported, lines can be drawn on screen or on the tablet (or both!); recognition can be immediate or delayed; JCRead@uclan.ac.uk

18. Supporting the writer (2) • Reviewing – computer can read back recognised text, child can read recognised or script text; spellings may be highlighted in recognised text – teacher controls • Editing – child can edit with rubber and pen on script, or with keyboard on recognised text JCRead@uclan.ac.uk

19. System States • Entry state • Recognition state • Edit state • DANGEROUS STATES • Pens that point • Cursors that confound • Spaces that stop JCRead@uclan.ac.uk

20. Interface Design (1) • Full writing screen • Ability to place new pages • Menus at the bottom • Haptic boundary preferred • Tablet matched to screen • Pen can be turned on and off JCRead@uclan.ac.uk

21. Interface design (2) • Video clip facility • Teachers screen • Assistant • Customisable • Training activities JCRead@uclan.ac.uk

22. And so…………… • The designs for a product for a small group of users, for a narrow application • Keyboard interface • Error repair • Speech recognition JCRead@uclan.ac.uk

23. Thank you Janet C Read University of Central Lancashire Preston Up North! England JCRead@uclan.ac.uk JCRead@uclan.ac.uk