Nvidia LightField display. [Image: Nvidia Corp.]
Computer scientist Douglas Lanman is on a mission to get more optical scientists excited about virtual reality (VR) and augmented reality (AR).
Lanman, director of display systems research for Facebook Reality Labs—formerly known as Oculus VR—said he rounded up his friends, acquaintances and researchers whom he admires when organizing the Frontiers in Optics/Laser Science 2019 conference track on simulations of reality. These “true believers” may be at their first optics conference, he said, but they are seeing that some of the best optics graduate students are getting job offers in the VR and AR fields, even though they aren’t hard-core gamers.
A passion for beautiful images
“Pretty much all of modern optics, from metamaterials to freeforms to computational optics, all of it is being leveraged for something I’m deeply passionate about, which is making beautiful images,” Lanman said.
According to Lanman, computational displays—an umbrella term for VR and AR units—must be optimized for the entire human visual system, not just the retina. Existing wide-field-of-view head-mounted displays (HMDs) are still a bit less sharp than the 20/20 standard for visual acuity. One pathway to improving these wide-field displays would use two or more optical paths to create a scene with higher resolution in the center than at the periphery. VR and AR displays had a 40-degree field of view 10 years ago, but manufacturers have pushed that to 90 degrees and beyond, Lanman said.
Designers of VR/AR systems also need to consider the combination of, and sometimes the conflict between, vergence (the movement of eyes in opposite directions to obtain a single image) and accommodation (the change in the eye’s optical power to focus near or far). Today, a typical VR headset has uniform focus across the scene. But Lanman said the first true computational display will “project” scenery onto one or more structured focal planes, instead of a flat surface, for more realistic depth of field.
Toward a “virtual personal assistant”
Today, many people hardly use their smartphones for actual phone calls—and someday, 90 percent of the function of AR glasses will not be as AR displays, said Henry Fuchs, the Federico Gil distinguished professor of computer science at the University of North Carolina at Chapel Hill, USA. But current AR headsets are heavy to wear all day and tiring on the eye muscles.
Fuchs, who has been studying graphics and rendering algorithms since the 1970s, envisions a “virtual personal assistant” with sensors that track a user’s body, not just his or her eyes, and is “situationally aware” of its surroundings. Such a device could show a physical therapist how a patient is doing in a home workout or display the names of attendees at a meeting. Of course, these huge data sets will create correspondingly large issues with privacy and security.
Creating realistic scenes
If you think a wide field of view in AR isn’t important, try walking around while looking through a couple of narrow cardboard tubes, said David Luebke, Nvidia Corp.’s vice president of graphics research. The challenges of building an AR display that people will want to use include keeping the wide-view device thin and light, with ultra-low latency and minimal power requirements. One technique recently developed at the University of California, Berkeley, USA, is ChromaBlur, which exploits the chromatic aberration of the human eye to create more realistic VR scenes.
Luebke stressed that AR technology must be inclusive, supporting users’ lens prescriptions and common vision disorders. In the near future, micro-OLEDs (micro-sized organic LEDs) could be incorporated directly into eyeglass lenses.
Floor demos
This year, VR and AR spilled out from the meeting rooms into the FiO+LS exhibit hall, where attendees lined up to test out the newest technology. Facebook Research Labs offered a chance to try try its Oculus Rift VR headset and hand controllers, which put users in charge of red and blue “swords.” (Some of the people waiting in line for the demonstration passed the time by playing the latest AR games on their smartphones.)
Circle Optics, a startup based in Rochester, NY, USA, demonstrated its Hydra camera, which gave wearers of a VR headset a 360-degree virtual view, including the wearer’s own back. Another startup, the Looking Glass Factory, demonstrated a non-headset approach: a holographic display, made of a thick block of glass, that shows multiple viewers a 3-D image without the need for any special glasses.