Interactive Systems Technical Design

1. Interactive Systems Technical Design Seminar work: Virtual Reality Teemu Laukkarinen Juha Mustonen Pekka Mäki-Asiala

2. Introduction - Definitions • Virtual Reality -> Virtual Worlds -> Virtual Environment • Artificially created believable impression of objects, environment or space that doesn’t exist in reality. • To measure the degree of virtuality we can use three gauges : • autonomy - modelling objects and processes • interaction - possibility to interact with the environment • presence - deceiving all senses to believe it is real • Immersion : • Loss of consciousness of the outside world; feeling of being surrounded by a completely other reality. • Immersion is psychological - relates to individuals.

3. Motivation - Where & Why • Entertainment • gaming industry • interactive virtual worlds • theme parks • Visualization • architecture, chemistry, CAD • arts and tourism • acoustics • Simulation & training • military, civillian (police, firefighters), • medicine, handicap aid • robots (remote control)

4. Implementation - needs/ways? • Implementing a virtual environment starts by defining needs. • What is the level of immersion? • Visual • Audible • Smell and taste - No technologies at the moment. • Tactile feedback - force feedback • Sense of balance - location and position detection

5. Visual (1/3) • Sight is the most important channel of sensory • Our eyes are : • accurate - real world details vs. computer generated resolution • accommotative - adapts to different distances • adaptive - lighting conditions • Fooling the eye • field of vision - 180o x 120o, must be covered totally • motion detection - perspective and texture changes • stereopsis - perseption of depth • Immersive VR displays • Head-Mounted Display (HMD) • Virtual Model Display (VMD) • Spatially Immersive Display (SID)

6. Visual (2/3) • HMD (Head-Mounted Display) • User wears a helmet that projects separate image for both eyes • Head orientation tracking important! • Relatively cheap • Field of view limited (resolution) • Uncomfortable to wear • VMD (Virtual Model Display) • Example: ImmersaDesk • Huge monitor, stereo vision (shutter glasses) • Possibly head tracking • Multi-user possibility • low immersion

7. Visual (3/3) • SID (Spatially Immersive Display) • User is surrounded physically by the display • CAVE-type setup • 3-6 ”walls” • Curved wall setup • one curved wall • curved overhead screen • sphere

8. Audible • We hear with our ears and with our body • 3D - sound is produced using • multiple speakers with/or dsp • headphones - HMD • Sensing the direction of sound • differences between sound pressure • delays - between ears • shape of our ears • Other factors • reflections, absorption - acoustics, dsp • visual and audible synchronization • doppler - is the object approaching?

9. Tactile & force feedback Very primitive ways compared to visual and audible feedback Cutaneous and Vestibular Senses (touch and balance) • Skin, joints, muscles, inner ear, internal organs • Tactile feedback helps to identify objects and shapes • Force feedback is essential part in developing many entertainment and industrial applications • Techniques • electrical - vibrating crystals • magnetical - changing magnetic fields • hydraulics - compressed liquid • pneumatics - sense of pressure • SMA (Shape Memory Alloy)

10. Location and position detection • Detecting location, movement and position of : • head - adjusting graphics, frames, delays - important • hands - data gloves (e.g. virtual surgery) • body - virtual suits, exoskeletons (e.g. entertainment) • Location trackers • performance can measured • resolution and accuracy, - measured value vs. real value • response time - update rates and latency • repeatability - variations between measurements • mechanical - accurate, low latency, restricts movement • magnetic - accurate, inexpensive, high latency • optical (camera + tracking point) - accurate, expensive, line of sigth • acoustic (ultrasound) - inexpensive, line of sight • inertia and non-inertia - inexpensive,

11. Applications - CAVE • CAVE (Cave Automatic Virtual Environment) SIGGRAPH 1992 • Cube-shaped room with floor, 3-6 walls, (optional ceiling) • Vision: • Shutter or polarization glasses high resolution CRT projectors • Sense of hearing: • Multi channel sound (n x speakers) • Sense of balance: • Head location and position detection • Multi-user option: • One room many participants • User is inside (immersion in reality): • User’s body is visible all the time

12. HUTCave • HUT (Helsinki University of Technology) www.tcm.hut.fi • Telecommunications and Multimedia Laboratory (TML) • 4-sided cave, three walls and a floor, • Hardware : • Main computer : Onyx2 InfiniteReality - SGI www.sgi.com • Projectors (CRT) : 4 x Electrohome Marquee 8500 LC • Crystal Eyes CE2 stereoglasses • Audio : • Pentium 4 Linux PC • Korg 880 D/A converter • 15 Genelec 1029A speakers • Tracking (magnetic tracking) : Ascension Motionstar • Input device : Surfman radio mouse

13. HUTCave - Architecture The CAVE room layout, side view. • Things to consider: • Ventilation and cooling • Power consumption • Lighting (relates to contrast) • Acoustics (absortion, echo…) • Wiring (hidden - maintenance) The CAVE room layout.

14. CAVE - Strengths • Comfortable for user • No need to wear heavy equipment • Turning the head is very natural • Very immersive • Multi-user option • The most popular solution for SID setup these days • Lots of info of ”how and why” • Compared to curved wall setups (possible exception - sphere) • Cheap • Many applications (universities, industry etc.)

15. CAVE – Problems • Expensive - not for consumer • Takes a lot of space - projection distance, wall size, operating system • Permanent installation - moving is not easy • Projector calibration (CRT) and seams of displays • Stereo vision with shutter glasses - Synchronization of screens • Full 6-sides - Wiring and air conditioning problems • Floor • Direct projection: Undesirable shadows • Back projection: Material of the floor cannot be the same as walls

16. CAVE - Fakespace video

17. Selected Industrial Players • http://www.fakespacesystems.com • Development and worldwide deployment and service of VR technology for business applications since 1988 • Products • WorkWalls • one or more large projection screens • stereographic images • WorkSpaces • CAVE, RAVE II • WorkDesks • Desks utilizing stereographic images • WorkTools • The CubicMouse, Pinch Gloves etc. • WorkStations • Value Added Reseller of HP and SGI

18. Rave II at Army Research Laboratory in Aberdeen, Maryland

19. ImmersaDesk R2

20. Selected Industrial Players • WorkTools • Pinch Gloves - Allows users to pinch and grab virtual objects or initiate actions with hand gestures - Detects contact between fingers • The CubicMouse - Allows specification of coordinates in virtual spaces - Three control rods, one for each axis (x,y,z), which can be pushed, pulled and rotated - Also the position and rotation of the controller itself can be taken into account VIDEO: cubicmouse_medical.mov

21. Selected Industrial Players • Fakespace’s customer examples • Manufacturing • US Army TARDEC/National Automotive Center (NAC) CAVEs for evaluating virtual prototypes and design problems • Oil and gas • World Oil / Murphy Oil Products used to speed and enhance geophysical and geological data interpretation • Education • Sun™ Center of Excellence for Visual Genomics CAVE used to immerse scientists in three-dimensional models of biological systems • Government • GM Defense Group CAVE, WorkWall and ImmersaDesk solutions used in designing and marketing of Light Armored Vehicles (LAV)

22. Selected Industrial Players • http://www.flogiston.com • Flogistation 2001 and 2002 • Features a "bubble-hood" display, motion base, hand controller, and a six-channel sound/vibration system • Originally developed as a training system for astronauts and funded by NASA • Has starred in a couple of movies, the most famous appearance in The Lawnmower Man

23. Flostation 2001

24. Selected International Research Groups and Projects (EVL) • Electronic Visualization Laboratory (EVL) - University of Illinois,Chicago - http://www.evl.uic.edu • A graduate research laboratory specializing in virtual reality and real-time interactive computer graphics • Research on • VR devices (home of the CAVE) • Develops compelling prototypes for improvement and reproduction by the commercial/industrial sector • VR Software (software libraries, toolkits) • VR Applications • Tele-immersion • “Collaborative virtual reality over networks“ • Networking • Tele-immersion as ‘application driver’ for developing high-performance broadband networks • Utilizing VR for artistic purposes

25. Selected International Research Groups and Projects (Presence-research.org) • Aims to offer up-to-date and relevant information and resources on (tele-)presence • Lists research labs, resources, on-going projects and people concerning VR and telepresence • News Archive • Conference dates

26. Selected International Research Groups and Projects (HIT Lab) • http://www.hitl.washington.edu/ (HIT Lab) • Research and development lab in virtual interface technology • Affiliated with the College of Engineering at the University of Washington in Seattle • Project examples • VR pain control • VR therapy for spider phobia • Motion sickness in VR environments • Virtual prototyping of medical robotic interfaces

27. Selected International Research Groups and Projects (HIT Lab) • VR pain control and therapy for spider phobia • Pain requires a conscious attention, being drawn into another world drain a lot of attentional resources, leaving less attention available to process pain signals • Technology derived from the similar spider phobia project • Especially used during woundcare of patients with severe burns

28. Selected International Research Groups and Projects (HIT Lab) • Since patients with severe burns often report re-living their original burn experience during wound care, SnowWorld was designed to help put out the fire • Patients fly through an icy canyon with a river and frigid waterfall. Patients shoot snowballs at snowmen and igloos (with animated impacts)

29. Researchers in Finland • Helsinki Arena 2000 • Virtual city • Elisa Communications & City of Helsinki • Lumeportti • Low-cost visualization environment • VTT

30. Helsinki Arena 2000 • Project started 1996 – one milestone 2000 • Aim to provide services also in the digital media • Leading city of the Europe when it comes to culture and services • Provide wider possibilities in interaction and communications

31. Helsinki Arena - Services • Panorama tour in centrum • 19 different places • Links to other worth-to-see - places

32. Helsinki Arena - Services

33. Virtual Helsinki - Services

34. Helsinki Arena - Technologies • WWW pages • Rich media • JAVA applets – IBM Hot Media • Panorama view – 360° • Zoom • VRML97 models – TRIDENT • Virtual telephone book • When traditional methods will fade out -> 3D Directory services competitive solution

35. Lumeportti • Project in VTT • Low cost visualization environment • Can be used for meetings, training and entertainment

36. Lumeportti - Technologies • Networked PCs and video projector • All the participants can interact on project data with electronic pens or other interaction devices.

37. Resources • Virtual Helsinki • http://www.virtualhelsinki.net • http://www.arenanet.fi • http://www.helsinkiarena2000.fi • http://www.trident3d.net • Lumeportti • http://mango2.vtt.fi:84/tte/projects/vr-cluster/kuvia/Lumeportti_taitto4.pdf

38. Future developments • A few years ago, Virtual Reality was treated as the immediate future by the • media, and then dismissed when technology could not evolve fast enough to be comfortable, affordable, and useful. • 3D user interfaces - monitors with shutter glasses • Home theaters • Immersive large displays, stereoscopic image, 3D sound • Mobile phones, wearable computing -> mixed reality

39. Thank you Questions?