Output Devices

Output devices enable the program to communicate its state to the environment. To express an information to a user, human senses must be stimulated. How this can be achieved is discussed below, focusing on vision.

2.1.2.1 Stimulating Vision: Interactive Displays

For a majority of VEs, the key aspect is the optical display, matching the importance of vision for people exploring an unknown environ-ment. Today, there are two important categories of immersive VR displays: large display screen setups and HMDs, i.e. small screens directly in front of the eyes.

For human vision, many effects come into play. There is no display today that can match the capabilities of the human eye. Resolution, contrast, absolute brightness, field of view, stereoscopic view with accommodation and motion parallax are features that are available individually, but it is not yet possible to integrate all of them in a single device. However, many effects can be handled to a satisfying extend.

The following figure gives an overview on important available display technologies and options.

10

2. HARDWARE DEVICES

HMDs Screens and Projections

Holographic Displays Volumetric Displays

– Rotating spiral screen

– PureDepth’s Multi-Layer LC Display – Rotating LCD

– Laser focusing on points in fog volume – IR laser detects dust in air and turns on visible laser when at correct position

Imaging technologies Projectors (rear / front projection) – LCD

– DLP – LED – Laser

– HDR (2 modulators + high luminance) Screens

– TFT/LCD – Plasma – OLED

– HDR (2 modulators + high luminance) Unconventional projection surfaces – Mist/fog/water ’curtains’

– Holoscreens (prisms)

– See through / opaque – Retinal Scanning Laser Display – Tiled displays (resolution, FoV) Stereoscopic displays with glasses

– Anaglyph (red-cyan / spectral comb: Infitec) – Polarization (linear / circular)

– Pulfrich effect (dark and light filter) – Diffraction grating (ChromaDepth) – Time interlaced (LCD shutter glasses) Autostereoscopic displays

– Lenticular lens vs. parallax barrier – Horizontal vs. complete parallax (lens array) – Light field displays

Figure 2.6: An overview of selected relevant technologies and options for (semi) immersive displays.

Figure 2.7: Anaglyph and polarized glasses are common technologies to realize cheap stereoscopic displays.

Many of these technologies can be combined. As an example, it is possible to build a stereoscopic or even an autostereoscopic HDR projection display, but to our knowledge it has not been constructed yet. A less useful example of an existing combination is a stereoscopic multi touch table. With the users standing next to the table, the large angle to the screen requires head tracking and two separate images for each observer. Furthermore, touch interaction can only occur on a single plane. Objects that appear above the screen surface are unsuited since a hand or arm will not be correctly occluded. It is hard to find an application to justify the amount of effort for this setup.

Other interesting developments in display technologies are transflex-ive and e-paper displays that work well in bright environment light situations. By reflecting the existing light of the environment, they consume very little power.

Overlapping images displayed on the screen.

The images switch so fast that without glasses the images seem to be blended.

time

16ms

projector 1: displays image for left eye alternating with a black image

projector 2: displays image for right eye alternating with a black image

glasses: are opaque for one eye at a time, controlled by an infrared shutter signal that is synchronized with the projectors

16ms 16ms 16ms

Figure 2.8: Illustration of a time interlaced stereo display.

Projection technology. The major drawback of projectors is their limited brightness. Projectors operate within their thermal limits, light bulbs are expensive and need to be replaced after a few thousand hours of life time. Lasers as light source offer advantages but are even

more expensive. However, this is about to change, as the laser life time increases significantly. LED projectors are now getting interesting, but still suffer from rather low light intensities.

Projection screen geometry. Most projection screens are flat, but also curved screens are used, e.g. in dome theaters or flight simulators.

Compared to a CAVE, no visible edges exist. Even arbitrary geometry may be used as a projection surface. With cleverly designed content, this may lead to impressive results, as e.g. projections on buildings show with arts or advertisement. Finally, neither the projector nor the screen have to be fixed. With appropriate tracking, the projection can be adapted to remain registered relative to the screen surface.

Projection screen material. Screen materials can be divided into reflective screens for front projection and translucent screens for rear projection. Different gain factors are available and can be used to increase the brightness, but at the same time restricting the viewing angle. When using polarization, many translucent screens work more or less well, whereas for front projection silver screens are essential to retain polarization. Holographic screens are holographic grated prisms on a translucent sheet, directly reflecting light from a projec-tion angle that can be quite different from the screen normal. Mist and fog can also be used as mostly flat screens or even volumetric screens.

Vergence-Accommodation conflict. Almost all current stereoscopic displays share a common problem: Accommodation is not han-dled correctly. While usually accommodation and vergence act to-gether, in screen based or HMD devices the eyes focus on a fixed screen distance rather than the 3D object distance, causing discom-fort and eye strain [RMWW94],[LIH07]. This is called the vergence-accommodation conflict.

3D glasses screen

real object virtual object

image seen by the left eye

image seen by the right eye

image seen by the left eye

image seen by the right eye

HMD displays

virtual object

image seen by the left eye

image seen by the right eye

Figure 2.9: Stereoscopic viewing (from left to right): the real stereoscopic effect, a stereoscopic display with screens and glasses and an HMD. The displays can provide correct vergence but fail to provide correct accommo-dation.

One possible solution is a fast switching lens and time inter-laced rendering of several depth layers [LHH^∗09]. It uses a stack of lenses, each consisting of a birefringent material and a fer-roelectric liquid-crystal polariza-tion modulator. It seems quite possible to eventually integrate such lenses in glasses and to build e.g. a CAVE with this tech-nology and fast projectors. A brighter display also reduces the problem, as the pupil gets smaller and the depth of field larger. Unfortunately, excessive costs make this simple idea for an enhancement infeasible for many projection setups.

Also, the retinal scanning laser displays have a related problem: Their imagealwaysappears in focus. Deformable mirror surfaces can pro-vide physical defocus. Additional software generated defocus cues like software depth of field blurring may be a solution when vergence can be measured [SSKS03], [SSSF04].

12

2. HARDWARE DEVICES

Holographic and Volumetric displays provide better or correct accom-modation and vergence cues [HGAB08], but need a lot of hardware effort. Also, they can only provide correct occlusion information for a single viewpoint at a time.

CAVE vs. HMD. CAVEs and HMDs are the primary choices for immersive VR setups. The following list shows the major differences.

In favor of the CAVE In favor of the HMD Tracking accuracy and latency

problems have less impact in a CAVE.

CAVE needs a large room, takes up lots of space.

Size perception & level of

immer-sion are better in a CAVE. HMDs are easy to transport.

The CAVE provides a large field of view.

HMDs are usually less expen-sive.

In the CAVE the user can see the own body.

HMDs consume a lot less power.

A high resolution can easier be realized in a CAVE.

HMD may need calibration be-fore each usage.

HMD is heavy and uncomfort-able.

Figure 2.10: Feature comparison of CAVEs versus HMDs.

High Dynamic Range Displays. HDR displays with constant illu-mination need two light modulators [See09]. Such a design needs lots of light and usually, most of the light is absorbed in the display.

The key for efficient HDR displays is a modulated light generation combined with an additional modulator, like a coarse LED matrix as background illumination combined with an LCD [SHS^∗04]. Espe-cially designed for HDR projection, a MEMS mirror array can be used to unevenly distribute the available light before it passes through the second modulator [HS08], [HSHF10]. Another option for an HDR display is to project an image on a modulated reflectance screen, like screens in e-books with e-ink, or just a paper print for a static image [BI08].

2.1.2.2 Output devices For Other Senses

Vision is arguably the most important sense to be stimulated in a virtual environment. However, it is reasonable to stimulate more senses than just vision in order to increase the level of immersion.

There is no clear definition of a sense. The following list shows the human senses that are commonly recognized as such, together with important technologies that can be used to stimulate each sense, more details follow below.

Human Senses Important Technologies Vision (sight) ⇒ Displays, see above.

Audition (hear-ing)

⇒

Audio (synthesized or prerecorded) sound or voice: electro-mechanical speaker, ear phones, wave field syn-thesis, HRTF

Tactition (touch) ⇒

Haptic devices, fan wind, mist/

water spray, Ultrasound Tactile Dis-play[HTNS09]

Gustation (taste) ⇒ Artificial aroma Olfaction (smell) ⇒ Olfactory device/gun Proprioception

(limb locations and motions and muscular force within the body)

⇒

Locomotion devices, walking simu-lators (2D treadmill, VirtuSphere), swimming simulator

Kinaesthesia (ac-celeration)

⇒

Motion platforms and haptic displays, handheld force display by nonlin-early perceived asymmetric accelera-tion [AAM05]

Equilibrioception (balance)

⇒ Influence on balance by strong mag-netic fields

Thermoception (temperature differences)

⇒

peltier hot and cold elements, electri-cal heating by current through resistor, infrared heat lamps

Nociception (pain)

⇒ Electrical shocks/stimulation Sense of time ⇒

-Figure 2.11: Human senses relevant to VR, with important technologies to stimulate them.

More senses exist, e.g. internal senses related to digestion. There are no devices that can perfectly stimulate even just a single human sense.

Even though relevant, the details of the senses and how they work, especially in combination with the brain, are out of scope.

Audition. Natural sound is a sum of monophonic sound sources.

However, distance and position of the sound source are important information for humans. The external ear and upper body act as a frequency filter. Also, the different positions of the ears lead in general to a time difference of the sound reaching each ear. Both effects are described by the Head Related Transfer Function (HRTF).

Convolving a virtual sound signal with the appropriate HRTF, the sound can be rendered and output with headphones for a 3D sound sensation. Using head tracking in a VE, the relative angle and distance of the sound source can be computed. However, sound transmitted through the body can not be simulated by that technique. Another approach is wave field synthesis, using a few dozen loudspeakers in an array. For a small target volume, the sound is approximately

14

2. HARDWARE DEVICES

reconstructed. The major downside is a rather large and expensive hardware effort.

Figure 2.12: Olfactory device with nose tracking. Photo courtesy of [YKNT03].

Olfaction. In general it is not possible to generate any odor by a combination of a few basic components. Usually, all odors to be displayed are produced individually and put in a different container.

An overview, also discussing recording and transmission of smells, is given in [DHL01].

It takes a long time to replace one smell by another for a whole room.

A solution for an olfactory display that is not attached to the user and allows some motion is shown in the image on the side. The smell is launched in the direction of the nose [YKNT03], [NNHY06].

Perceptual Illusions. The human senses are not perfect measure-ment instrumeasure-ments. Studies show a perception mismatch in VEs, e.g.

objects appear larger or walked distances shorter than they should.

Stimulating the senses in a clever way, a number of different side effects can be exploited to trick the perception. Senses can be ma-nipulated in an unconscious way, as in walk redirection [RKW01] or exaggeration of head rotation [LFKZ01]. In some cases, such tech-niques allow to bypass or reduce hardware limitations.

2.1.3 Input Devices

In document Visual Computing in Virtual Environments (sider 13-18)