3-D TV and Movies: Exploring the Hangover Effect

Stereoscopic imaging fools our eyes into seeing three dimensions in a flat image. The trick is an old one, presenting our two eyes with different images that we fuse into a single three-dimensional view. It's intended to simulate our natural binocular vision. When done well, it can be very convincing.

Now, electronics companies have downscaled the technology from theaters to home televisions. Look through a pair of 3-D glasses properly synchronized with a good flat-panel 3-D TV, and you can see animated characters like those in Monsters vs. Aliens come to life in three realistic dimensions.

Each eye sees a slightly different two-dimensional view of the world, which binocular vision fuses into a three-dimensional image. But stereoscopy only fools part of our visual system. Our ancestors evolved alternative ways of sensing the location of food, predators and other objects because they needed to judge such distances in order to survive.

One such method is by tracking accommodation—how the eye adjusts its focus to minimize blur and produce a sharp image on the retina. Others include observing how near-field objects obstruct more distant ones, comparing observed sizes with expected ones, and noting any obscuration by atmospheric haze. Our autonomic nervous system does that without conscious thought.

When different sensory systems yield varying results, people may perceive nausea, headaches or other symptoms. That is what causes motion sickness. The inner ear detects movement, like the rocking of a boat, which the eyes don't recognize because they are looking at another part of the boat. Our bodies can ignore minor differences for awhile, but eventually some of us feel seasick.

When we view stereoscopic images, the eyes converge to a point at the apparent distance of the 3-D image, an eye movement called vergence, but they accommodate to focus the plane of the display on the retina. The resulting visual conflict can cause eyestrain, the severity of which depends on individual sensitivity and the strength and duration of the discrepancy.

Early stereoscopic systems

The idea of stereoscopy grew from the discovery of perspective, but it didn't blossom until the invention of photography. In 1849, David Brewster built the first stereoscopic camera with a pair of lenses side by side to record pairs of photos. Viewed through a pair of eyepieces, it gave the illusion of depth. The first stereo images were still pictures; however, later in the 19^th century stereo-pair photos were mounted on wheels and flipped before viewers to give the illusion of motion as well as depth.

Another mid-19^th-century invention was the two-color anaglyph process, which records image pairs in different colors and then projects the two colored images onto a screen. Viewers wearing glasses with one red lens and one green lens see the projected image in 3-D. Printed anaglyphs followed, and they have been used in cartoons and images of the Martian landscape. Anaglyph 3-D was first tried in movies in the early 20^th century, using either a pair of projectors with separate films or color overlays printed on a single reel of color stock.

In the 1930s, Edwin Land developed polarization-based 3-D. He was seeking markets for his new mass-produced polarizing sheets. Instead of overlaying images in different colors, he superimposed images in orthogonal polarizations for viewing in glasses with Polaroid filters mounted in orthogonal directions. The 3-D effect was attractive, but it required synchronizing a pair of cameras and projecting onto a silver screen to preserve the polarizations.

The advent of television stimulated the first big 3-D film boom in the 1950s. Seeking to lure customers away from their home sets, the movie industry turned to a color version of Polaroid's 3-D process. The first big 3-D hit came in November 1952. It was an otherwise mediocre action film called Bwana Devil. By May 1953, Polaroid was making six million pairs of polarizing glasses each week.

But the boom didn't last. The two projectors could easily drift out of synch, leaving the audience with hangover-like headaches. Many never came back, even when anaglyph films and a single-projector polarization process reached theaters. 3-D seemed doomed to niche audiences.

Virtual reality and cybersickness

Digital technology almost launched the next 3-D boom in the early 1990s—but it collapsed after people developed serious problems with nausea and disorientation.

The idea of virtual reality emerged in the mid-1980s, powered by steady improvements in computer systems and imaging. In the early 1990s, Sega, then a major maker of video games, decided to make an electronic virtual-reality game. Their design was state of the art, with computers driving a pair of liquid-crystal displays built into goggles worn by players. Internal gyroscopes would track player movement. Sega announced plans for a big introduction.

Unfortunately, when SRI International, a California consulting firm, began testing prototype goggles, people got sick. A key problem was the frame rate. The best computer chips then available couldn't generate frames fast enough to keep up with what people expected when they moved their heads. What users saw in the goggles clashed with motion sensed by the vestibular system of the inner ear. The frame rates were 4 to 12 per second and they "turned your stomach very quickly," says Tom Piantanida, a now-retired member of the SRI team. He calls that range "the barfogenic zone."

Putting images so close to the eye also creates visual problems because the eye adapts to the artificial world. "I wore it for eight hours, and when you took it off, the world didn't move as you anticipated," Piantanida recalls. "The image surface was two inches from the eye. The perceptual surface was 2 feet. That's where the disconnect occurred." Sega pulled the plug on their marketing plans, but they never admitted that the SRI results were a decisive factor.

The effect wasn't limited to Sega's design; they were inherent to head-mounted virtual-reality displays. In a separate study, Mark Mon-Williams, then at Caledonian University in Glasgow, found that a mere 10 minutes cycling (on stationary bicycles) through a virtual world induced symptoms that included blurred vision, headaches, nausea, tired eyes and motion sickness in 12 of 20 volunteers. He warned against installing such systems in workplaces or entertainment centers.

Virtual reality is particularly tough because motion can create a mismatch between the user's real world and the virtual reality. "The better the equipment, the worse the problem," says Paul DiZio, who studies cybersickness at Brandeis University. "It's a fundamental paradox of virtual reality."

With screens close to the eye and plenty of stereo content, Sega hit a "worst-case scenario," says Marty Banks, a vision researcher at the University of California at Berkeley. The system produced a visual conflict between the eye's focal accommodation and vergence, the turning of the eyes in opposite directions to point at an object that measures its distance by binocular vision. The mismatch is largest when the display is close to the eye and the stereo effect is large, so it is a big problem for head-mounted displays. It's much smaller when the display is far from the eye, making theaters a much better 3-D venue.

Digital technology and 3-D movies

Rapid progress in digital imaging launched the current 3-D boom. Computer graphics created a new generation of content, and the new digital projectors that have replaced film in many theaters readily lend themselves to 3-D screening. Three approaches to 3-D screening are in use—circular polarization, a six-color variation on the old anaglyph system and shutter glasses.

Circular polarization isolates the two eye channels better than the linear polarization used in the Polaroid system, and it limits crosstalk when viewers tilt their heads. The RealD system generates 144 frames per second, alternating right and left views, each of which is repeated three times before moving to the next frame, thus maintaining the old 24 frame/second film standard.

Light from the projector lens passes through a liquid-crystal polarizer, which switches between two circular polarizations 144 times per second, synchronized to the frame rate. A silver screen directs the light to viewers wearing inexpensive passive glasses with lenses of opposite circular polarizations. MasterImage3D offers a similar system that uses a filter wheel to switch between circular polarizations.

An alternative approach to circular polarization 3-D for smaller venues is covering a large-screen television with a thin array of circular polarizers, so that half the pixels are polarized for the right eye, and the other half for the left. This adds to the cost but allows the use of inexpensive passive glasses. Some British pubs lure patrons by showing sports events in 3-D, and the systems are starting to appear in U.S. sports bars as well.

The innovation of the Dolby 3-D system is a six-color anaglyph system that codes full-color images. It is based on a 2-D system that splits light from a single lamp into three colors, each of which is reflected by a separate DLP chip and combined into a full-color image. Instead of using broad color filters, the anaglyph version uses two sets of dichroic filters that select separate thin slices of the red, green and blue bands—one for the right eye and one for the left. Viewers wear glasses with matching sets of filters for their right and left eyes. The dichroic filter glasses cost much more than polarizing ones, so they must be cleaned and reused after each viewing, but the system does not require a silver screen.

Shutter glasses use active LCD filters that switch off and on in phase with a digital projector displaying alternating images for the right and left eyes. The switching is so fast that viewers don't notice any flicker. The XpanD system uses this approach, which suppresses ghosting when images intended for one eye leak into the other. The main drawback is that the glasses cost about 100 times more than passive polarizing glasses.

Cinema has big advantages over virtual reality goggles. People sit and stare at the theater screen, reducing conflicting motion cues that occur when they move about wearing virtual reality goggles, says DiZio. Furthermore, movie screens are so far from the viewer that the eye shifts its focus little for 3-D objects that appear a reasonable distance in front of or in back of the screen.

That doesn't exempt movie viewers from 3-D side effects. Individuals vary widely in their responses. Five to 10 percent of people have visual defects such as amblyopia that prevent them from seeing depth stereoscopically, so they see 3-D as flat and don't experience any issues. Others are unusually sensitive to visual discrepancies. The intensity of the content also makes a difference; extreme action that zooms back and forth from the viewer's lap to infinity can strain eyes even in the best of theaters. And poor operation or equipment failure can make 3-D displays slip out of synch and into the headache zone.

Television isn't the movies

Current 3-D TVs sold for home use in the United States are essentially high-end digital televisions with the added feature of being able to display 3-D when used with shutter glasses. The hardware resembles a miniature 3-D theater. A radio or infrared link synchs the glasses with the display, so the right lens switches on to show a right-eye frame, and the left lens switches on to show the left-eye frame. The shutter glasses cost around $150.

People watch television differently than they watch movies in the theater. Many television viewers multitask, switching their eyes back and forth between screen and homework, knitting, puzzles or some other activity. Special glasses make that difficult by attenuating background light. Glasses also get in the way of the usual home social interactions, such as turning to other viewers and talking about characters, plot or whatever else is being shown on the screen. Those without glasses see 2-D images on the screen that are ghosted and often unwatchable, making it hard to casually step into the room and glance at a show.

Home viewing also poses challenges for producers designing 3-D effects. What viewers see depends on their positions relative to the set. "You just don't have control over the environment in a home," says Banks. People sit too close, off to the side, or on the floor. They may lean their heads to the side. Background illumination is much brighter than in a theater.

The smaller viewing distance in homes is crucial because of the vergence-accommodation conflict. Simply put, your eyes don't have to adjust as much if a 3-D monster jumps halfway toward you from a screen 10 meters away as they do if the monster leaps from a distance of two meters.

Experiments at the Japanese Broadcasting Corporation (NHK) showed the importance of conflicts at television viewing distances by comparing visual fatigue in people watching 2-D and 3-D televisions. In 2004, Masaki Emoto and colleagues found that viewing 3-D images from a screen 1.2 m away caused severe visual fatigue in volunteers. The following year, Emoto and other colleagues found that jumps in the apparent distance of 3-D objects also contributed to visual fatigue. Other work also suggested that 3-D TV could cause eyestrain and headaches, although many details remain unclear because the effects are hard to measure.

Little word of such problems reached the public until April, when reporters uncovered a Samsung document titled "Viewing TV using the 3-D function." It includes some sensible-sounding warnings, including a recommendation to watch from a distance at least three times the screen's height, and one to wear the 3-D glasses when watching 3-D. Yet oddly it also warns not to put a 3-D TV "near open stairwells, cables, balconies or other objects that can be tripped over, run into, knocked down, broken or fallen over." Its most bizarre warning, and one that was widely reported on the web, is that pregnant women, the elderly, and people who were sick, overtired or drunk should not watch 3-D.

Samsung was not alone. A Sony document on a transmitter sending synchronization signals from 3-D sets to shutter glasses, recommends that "all viewers take regular breaks while watching 3-D video images." More unsettling, it urges parents to talk with their pediatrician or ophthalmologist before letting young children watch 3-D TV.

Asian companies have a reputation for caution, but why issue such broad warnings? Sony did not respond to queries, and Samsung never followed up on an interview request. However, in April, the website TechRadar.com quoted a Samsung spokesman as saying that their main concerns were the few percent of people who were unable to see 3-D properly, along with young children, who have small eye separation. As for the sick and pregnant, the warning was a precautionary measure to avoid overexciting these individuals.

Visual uncertainties

Researchers think some warnings may be overblown, but many uncertainties remain. So-called cybersickness, which includes nausea and disorientation, is a well-documented problem in virtual reality and simulation systems. A 2004 survey by the Australian Defense Science and Technology Orientation reported that flight simulators left some users dizzy, disoriented and unsteady on their feet—sometimes for hours. Mismatches of movement sensed by the eyes and inner ear are known to cause motion sickness, and DiZio says that fundamental questions linger because we still don't understand how different sensory systems interact.

Many details of visual side effects are still being worked out. In 2008, Banks's group confirmed that conflicts between vergence and accommodation can cause eyestrain. In an unpublished new study, they found that large visual jumps can cause more discomfort than had been expected at long distances. Volunteers viewed displays at three distances—10 m, 77 cm and 40 cm—and they observed that 3-D images seem to jump from one distance to the next.

Those jumps correspond to 1.2 diopters—enough for a monster to leap from a movie screen to within arm's reach. That's intense enough to cause visual discomfort, and, as expected, the discomfort was most intense up close. However, even at 10 m, it was worse than expected and potentially relevant for movie producers. Longer viewing distances still protect the viewer better from the discomfort caused by vergence-accommodation conflict than short ones, but the new results show the difference is not as great as had been thought.

The duration of sustained 3-D effects, as well as their intensity, contribute to eyestrain, but the thresholds depend on a combination of the two factors, and they vary widely among individuals. Amusement parks such as Disney limit their rides to half an hour to prevent motion sickness, and some 3-D films are also short. Despite the long length of Avatar (162 minutes), the movie only showed strong effects for only brief intervals. Most viewers seemed happy with the results, but some suffered headaches and nausea. The reason why remains unclear.

Good and bad 3-D

Stereo quality is also crucial to the 3-D experience; the absence of it can cause nasty headaches. Avatar is widely considered an example of good 3-D, particularly because of its moderation in effects and the care that was taken in production. Those qualities are far from universal. "Anyone who thinks stereo is easy can get a couple of cameras and start practicing the art," says John Merritt, a consultant in Williamsburg, Mass., U.S.A. Although it seems simple, because you've got two eyes and two cameras, Merritt explains, there are plenty of pitfalls. Too many untrained producers charge ahead without realizing their ignorance.

To improve 3-D quality, the 3-D@Home consortium, which promotes 3-D TV, has posted recommendations from Avatar director James Cameron and others. For example, Rob Engle, who supervised the 3-D adaptation of The Polar Express, warns that quick cutting between objects at different depths can cause eyestrain. He urges care in using strong 3-D effects because they break the narrative arc.

Computer animation has been successful in the 3-D realm because it gives more control over image depth and quality than live action, says Juan Reyes, chief technology officer at BluFocus in Toluca, Calif., U.S.A. Viewers prefer 3-D animation, says Paul Gray, director of TV electronics research for the consulting firm DisplaySearch. "I loved Toy Story in 3-D, yet sitting in the same cinema I had eyestrain with Avatar after 40 minutes."

Live action is trickier. Early 3-D sports broadcasts showed disembodied limbs or heads when spectators stepped before the cameras; now, Reyes says, producers buy seats in front of the camera to prevent disorienting intrusions. Care is vital in selecting shot depth. Crowds are best viewed from a distance, with shallow 3-D; close-up 3-D in a crowd or action scene can be disorienting because body parts and nearby objects appear and disappear, especially with fast motion. The director and camera crew must worry how to record scenes properly with two cameras. Reflections can be tricky. "If you're filming someone wearing a piece of jewelry, it can reflect light into one lens but not into the other," leaving the 3-D image without depth, says Reyes.

Proper setup of 3-D equipment is surprisingly easy to bungle. Reversals of right and left eyes are common, although they should be obvious because they make faces look hollow. Visiting corporate labs, he found that nobody in one lab had noticed their right and left channels had been reversed for six months.

Children's vision

The toughest issue raised about 3-D is its potential long-term impact on the developing vision of young children. There's pretty good evidence that sitting close to 2-D televisions increases the likelihood that children will become myopic, says Banks. He can think of some reasons why 3-D might impact children more than 2-D, and others why it might affect them less. Banks isn't convinced that there is a problem, and many people in the field seem to agree.

However, the lack of knowledge worries Mark Pesce, a virtual reality pioneer familiar with the Sega system. He likes 3-D, and he wants for it to be safe, but after his experience with head-mounted displays, he isn't convinced that it is. Marc Lambooij, who studied 3-D effects in the Human Technology Interaction group at the Eindhoven University of Technology, thinks that too little research has been done on long-term effects of 3-D TV on the vision of children and others to introduce it commercially. He wants to see norms for content and guidelines for viewers similar to those for current 2-D TVs.

The great unknowns

One thing that the rush to 3-D entertainment has made clear is how little we know about stereoscopic vision. Researchers estimate that 5 to 10 percent of the population can't perceive stereo. However, hard numbers are as unobtainable as the number of viewers made ill by Avatar. We also don't know what makes some people more susceptible to 3-D eyestrain than others, just as we don't fully understand the roots of motion sickness.

The 3D@Home Consortium wants to answer those questions. It has established a human factors study group to look at health-related issues. After collecting information on past studies, they will decide how to resolve the inevitable uncertainties, says Paulette Pantoja, CEO of BluFocus and a member of the consortium. One way to help resolve questions about abnormalities in binocular vision would be to incorporate a screening test into 3-D TVs, suggests Banks.

So far, sales of 3-D TVs have been slow, but that does not seem to be related to worries about visual effects. When the Cable & Telecommunications Association for Marketing surveyed hundreds of viewers who had tested 3-D sets, they found their top three reasons for not buying were price, the need for special glasses, and the scarcity of 3-D programs.

Developers are working on systems that don't require glasses, and Toshiba has announced one such system. But Reyes doesn't expect that they will be ready for widespread use for at least five to seven years. Current approaches to glasses-free 3-D would restrict where viewers could sit, and they would not eliminate the vergence-accommodation disparity at the root of visual problems.

The only way to avoid that disparity is with true three-dimensional images. According to Michael Bove of the MIT Media Laboratory, holographic video "will ultimately prove practical and will provide the ultimate television viewing experience." Late last year, University of Arizona researchers demonstrated the first truly holographic telepresence system. But its low resolution and speed of two seconds per frame mean that holographic 3-D TV is still a long way off.

For now, we're stuck with 3-D technology that fools our eyes and risks triggering biological warning systems. Like amusement park rides, 3-D entertainment can be fun in moderation. However, evolution has had millions of years to shape our eyes to see the natural world comfortably and well in three dimensions. Our 21^st century technology has a long way to go before it can match that performance.

---

Jeff Hecht is a science and technology writer based in Auburndale, Mass., U.S.A.

References and Resources

>> 3D@Home Consortium
>> J. Barrett. "Side Effects of Virtual Environments: A Review of the Literature, Australian Defense Science and Technology Organization," DSTO-TR-1419.
>> M. Mon-Williams et al. "Binocular vision in a virtual world: visual defects following the wearing of a head-mounted display," Ophthalmological and Physiological Optics 13, 387-91 (1993).
>> V.K. McElheny. Insisting on the Impossible: The Life of Edwin Land, Inventor of Instant Photography, Perseus Books, Reading, Mass., U.S.A., 1998.
>> M. Emoto et al. "Changes in fusional vergence limit and its hysteresis after viewing stereoscopic TV," Displays 25, 67-76 (2004).
>> M. Emoto et al. "Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television," J. Display Technol. 1, 328-40 (2005).
>> D.M. Hoffman et al. "Vergence-accommodation conflicts hinder visual performance and cause visual fatigue," J. Vision 8(3) 33, 1-30 (2008).
>> M. Lambooij et al. "Visual discomfort and visual fatigue of stereoscopic displays: a review," J. Imag. Sci. Tech. 53(3), 1-14 (2009).
>> P.-A. Blanche et al. "Holographic three-dimensional telepresence using large-area photorefractive polymer," Nature 468, 80-3 (2010).
>> M. Lambooij et al. "Measuring visual fatigue and visual discomfort associated with 3-D displays," J. Soc. Inf. Disp. 18, 931-43 (2010).