Interaction in Virtual Environments

1. Interaction in Virtual Environments Benjamin Lok This Lecture contains notes created by Doug Bowman Virginia Tech Sherman & Craig, pp. 332-363

2. 3D Interfaces • We live in a 3D world • Natural interfaces are better • Therefore 3D interfaces will be the ultimate • What’s wrong with the above? • Natural interfaces aren’t always better! • Making the interface simple (thus unnatural) often aids performance • Constrains movement • Limiting possible actions • Depends on application and goal of the user interface • Surgery simulation • Military simulation (general vs. soldier training) • Architecture, education, product design • Video games

3. 3D Interfaces • What we really want are enhanced interfaces • Give us powers we don’t normally have • Flying, x-ray vision, teleportation, undo, etc. • Be careful we don’t become overzealous • Air traffic control 3D display • Library interfaces using a books on shelves (what is it good for? What is it poor for?) • Hurts performance • Study results: 3D Bar charts don’t help • So what is helped by 3D?

4. Good 3D • Social interfaces + 3D can be very powerful • MMORPG (EveQuest) • ActivedWorlds • The Sims Online • Experiences • Art gallary • 3D Desktops (Mac’s latest) • Office metaphors did not take off (BOB, Task Gallary) • 3D Webbrowsing. Sure you can arrange 16 web pages spatially, but why? • Compromises to provide 3D interfaces might be undermine usability • Think RTS games • Discussion: Is the interface holding back 3D?

5. 3D Interfaces • Use occlusion, shadows, perspective carefully • Improves use of spatial memory (Ark ’98) • Distracting and confusing • Minimize navigation steps • Keep text readable (good contrast, 30 degree tilt max) • Simple user movement (why lock to a floor?) Descent vs Quake • Prevent Errors (put in guides to help) • Simplify object movement (connecting two parts, for example, can be abstracted… most of the time) • Organize groups of items into alignments that facilitate visual search and recall (allow user choice)

6. 3D Interface Development • Developments that show promise: • 3D sound • Stereo display (Ware and Frank ’96) • Haptic feedback (mouse) • 3D can help by: • Provide overviews to see big picture • Rapid teleportation (context shifts) • Zooming (aid disabled) • Multiple coordinated views (3dsmax) • 3D icons can represent abstract or recognizable concepts

7. Teleoperation • Combines: • Direct Manipulation • Process Control • Human operators control physical processes in complex environments • Example applications: Mars rover control, flying airplanes (Predator), manufacturing, medicine (surgery) • Supervisory control (Sheridan ’92) • Different levels of human control (automation)

8. Teleoperation • Direct Manipulation Issues • Adequate feedback (data quality, latency (transmission and operation delays), incomplete, interference) • Presence • Point and click or more natural interaction vs. typing • Example project: Nanomanipulator (show video)

9. VR Interaction • Trying to simulate reality or an experience • Training, Learning, Exploring • Expensive • Dangerous • Logistically Difficult • Best interaction? • Flight simulators (they can cost $10 mil, but that’s still a good deal!) • Why? • Why do video game flight sims not cut it? (only $40!) • Okay, we have monitors that show 3D worlds, what else do we possibly need?

10. Why 3D interaction? • 3D / VE apps. should be useful • Immersion • Natural skills • Immediacy of visualization • But, current VE apps are • Not complex interactively • Have serious usability problems

11. Spatial input Lack of constraints Lack of standards Lack of tools Lack of precision Fatigue Layout more complex Perception What makes 3D interaction difficult? You’ve seen this in creating Project #2!

12. Let’s look at this example (ISAAC)

13. Universal interaction tasks • Navigation • Travel – motor component • Wayfinding – cognitive component • Selection • Manipulation • System control

14. Goals of interaction design • Performance (efficiency, accuracy, productivity) • Usability (ease of use, ease of learning, user comfort) • Usefulness (users focus on tasks, interaction helps users meet system goals)

15. Artistic approach Intuition about users, tasks Heuristics, metaphors Aesthetics Adaptation Scientific approach Formal analysis Formal evaluation Performance requirements Philosophies of interaction design

16. Travel • Motor component of navigation • Movement between 2 locations, setting the position (and orientation) of the user’s viewpoint • Most basic and common VE interaction technique • Used in almost any large-scale VE

17. Types of Travel Tasks • Exploration • Travel which has no specific target • Build knowledge of environment • Search • Naive: travel to find a target whose position is not known • Primed: travel to a target whose position is known • Build layout knowledge • Move to task location • Maneuvering • Travel to position viewpoint for task • Short, precise movements

18. Steering metaphor • Continuous specification of direction of motion • Gaze-directed • Pointing • Physical device (steering wheel, flight stick)

19. Target-based metaphor • Discrete specification of goal • Point at object • Choose from list • Enter coordinates • E.g. Google Earth

20. Route-planning metaphor • One time specification of path • Place markers in world • Move icon on map • Manual manipulation of viewpoint • “Camera in hand” • Fixed object manipulation

21. “Natural” travel metaphors • Walking techniques • Treadmills • Bicycles • Other physical motion • VMC / Magic carpet • Disney’s river raft ride • Simulation of flying

22. Technique classification Target specification Route specification Continuous specification Start to move position Indicate position velocity Travel acceleration Indicate orientation Stop moving

23. Alternate Technique classification gaze-directed pointing physical props Direction/Target Selection gesture slow in, slow out physical props Velocity/Accel. Selection Travel start/stop buttons automatic start/stop constant movement Conditions of Input Ontologies by Bowman99 and Hand97

24. Evaluation results • Steering techniques have similar performance on absolute motion tasks • Non-head-coupled steering better for relative motion • “Teleportation” can lead to significant disorientation • Environment complexity affects information gathering • Travel interaction technique and user’s strategies affect spatial orientation

25. Evaluation results – 3 • Manipulation-based techniques efficient for relative motion • Manipulation-based techniques not requiring an object efficient for search, but tiring • Steering techniques best for naïve and primed search • Map-based techniques not effective in unfamiliar environments, or when any precision is required

26. Myths • There is one optimal travel technique for VEs. • A “natural” technique will always be better than another technique. • Desktop 3D, workbench, and CAVE applications should use the same travel ITs as HMD-based VEs.

27. Design guidelines • Make simple travel tasks simple (target-based techniques for motion to an object, steering techniques for search). • Provide multiple travel techniques to support different travel tasks in the same application.

28. More design guidelines • Use transitional motions if overall environment context is important. • Train users in sophisticated strategies to help them acquire survey knowledge. • Consider integrated (“cross-task”) ITs if travel is used in the context of another task (e.g. manipulation).

29. Wayfinding • Cognitive process of defining a path through an environment using and acquiring spatial knowledge • 6DOF makes wayfinding hard • Human beings have different abilities to orient themselves in an environment • Observing wayfinding as a decision making process

30. Wayfinding tasks • General, explorative search Search without target • Naïve searchtarget position unknown • Primed searchtarget seen before (known) • Specified trajectory movementPredefined path

31. Cognitive Map • During wayfinding, a person makes use of three kinds of knowledge to built up a cognitive map of the environment: • Landmark knowledge • Procedural knowledge • Survey knowledge

32. Reference frames • Egocentric reference frame: position, orientation, movement of object with respect to position and orientation of the: • eyes • head • body • Exocentric reference frame:position, orientation and movement are defined in coordinates external to body • object shape • object orientation • object motion

33. Travel technique effects • Steering technique with good strategy helps spatial orientation • A good travel technique will integrate aids to wayfinding • Jumping between points disturbs spatial orientation

34. Support of spatial knowledge acquisition • Allow a wide field of view • Provide motion cues for judging depth and direction of movement • Audio could enhance visual spatial perception • Support sense of presence: it could strengthen the construction of a cognitive map

35. Support of spatial knowledge acquisition • Design legible environments - allow the user to easily see the spatial organisation of an environment, enabling the establishment of a cognitive map. • Divide a large-scale environment into parts with a distinct character

36. Support of spatial knowledge acquisition • Create a simple spatial organisation in which the relations between the parts are clear • Support the matching process between the egocentric and exocentric frames of reference by (visual) cues, including directional cues

37. Support of Spatial Knowledge Acquisition • Use real-world wayfinding principles to build up your environment • Natural environment principles • Urban design principles [Lynch] • Architectural design principles • Artificial cues

38. Examples of wayfinding aids

39. Map usage guidelines • Provide you are here marker • Provide grid • Choose either north-up or forward-up map, depending on task • Example: World-in-Miniature

40. Manipulation Techniques • Wand-based • Go-Go • Ray Casting • HOMER (Hand-centered Object Manipulation Extending Ray-casting)

41. Two Handed and Body-Centered Interaction • What can you do with two hands? • What if you use your body as a reference point? • Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin (1997). Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Angeles, CA. (133K pdf version)

42. Two Handed and Body-Centered Interaction • What can you do with two hands? • What if you use your body as a reference point? • Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin (1997). Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Angeles, CA. (133K pdf version)

43. Other Manipulation Approaches • Tablet/hand-held • Pinch Glove