3D Interaction Techniques for Virtual Environments

1. 3D Interaction Techniques for Virtual Environments Doug A. Bowman

2. Terminology • Interaction Technique (IT) – method for accomplishing a task • 3D application – system that displays 3D information • 3D interaction – performing user actions in three dimensions

3. ITs Didn’t we already cover input devices? User interface software Input devices System Software Output devices User

4. Video example: ISAAC

5. Universal interaction tasks • Navigation • Travel – motor component • Wayfinding – cognitive component • Selection • Manipulation • System control • Symbolic input

6. Selection & Manipulation • Selection: specifying one or more objects from a set • Manipulation: modifying object properties (position, orientation, scale, shape, color, texture, behavior, etc.)

8. Goals of selection • Indicate action on object • Query object • Make object active • Travel to object location • Set up manipulation

9. Selection performance • Variables affecting user performance • Object distance from user • Object size • Density of objects in area • Occluders

10. Common selection techniques • Touching with virtual hand • Ray/cone casting • Occlusion / framing • Naming • Indirect selection

11. Enhancements to basic techniques • Arm-extension • Mapping • “Reeling” • 2D / 3D World in Miniature • Select iconic objects

12. Go-Go implementation • Requires “torso position” t - tracked or inferred • Each frame: • Get physical hand position h in world CS • Calculate physical distance from torso dp = dist(h, t) • Calculate virtual hand distance dv = gogo(dp) • Normalize torso-hand vector • V. hand position v = t + dv *(h-t) (in world CS)

14. Selection classification graphical tactile audio Feedback object touching pointing indirect selection Object indication Selection button gesture voice Indication to select

15. Evaluation: Selection Task • Ray-casting and image-plane generally more effective than Go-Go • Exception: selection of very small objects can be more difficult with pointing • Ray-casting and image-plane techniques result in the same performance (2DOF) • Image-plane technique less comfortable

16. Implementation issues for selection techniques • How to indicate selection event • Object intersections • Feedback • Graphical • Aural • Tactile • Virtual hand avatar • List of selectable objects

17. Implementation issues for selection techniques • How to indicate selection event • Object intersections • Feedback • Graphical • Aural • Tactile • Virtual hand avatar • List of selectable objects

18. SIGGRAPH 2001 tutorial 자료

19. Common selection techniques • Simple virtual hand • Ray-casting • Sticky finger (occlusion) • Go-go (arm-extension)

20. Simple virtual hand technique • One-to-one mapping between physical and virtual hands • Object can be selected by “touching” or intersecting v. hand with object

21. Ray-casting technique • “Laser pointer” attached to v. hand • First object intersected by ray may be selected • User only needs to control 2 DOFs • Empirically proven to perform well

22. Ray-casting implementation • Naïve: intersect ray with each polygon • Parametric equation • Only consider intersections with t > 0 • Better: transform vertices (or bounding box) to hand’s CS • Drop new z coordinate of every vertex • Ray intersects polygon iff (0,0) is in the polygon • Count the number of times the polygon edges cross the positive x-axis

24. Occlusion technique • Image-plane technique - truly 2D • Occlude/cover desired object with selector object (e.g. finger) • Nearest object along ray from eye through finger may be selected

25. Occlusion implementation • Special case of ray-casting technique • Must consider position of eye/camera • Can use 2nd ray-casting algorithm; requires special object ß ß

26. Go-Go technique • Arm-extension technique • Like simple v. hand, touch objects to select them • Non-linear mapping between physical and virtual hand position • Local and distant regions

27. Go-Go implementation • Requires “torso position” t - tracked or inferred • Each frame: • Get physical hand position h in world CS • Calculate physical distance from torso dp = dist(h, t) • Calculate virtual hand distance dv = gogo(dp) • Normalize torso-hand vector • V. hand position v = t + dv*th (in world CS)