Imagine if each one of the pixels on a screen wouldn’t have just one color, and that such pixels could seem to be of a different color depending on the angle from where you look at it. According to Reddit user solinvicturs21, this is the principle of holography which creates static prints that appear entirely three-dimensional, and it’s what a light field display does for moving images. It’s called a light “field” because for a video camera to capture a light field, each pixel must capture the light striking from every direction at each pixel location rather than just the light coming from straight ahead.
Light Field Display is Like Looking through the Window
What we are beginning to call light field displays today are very limited in their capabilities regarding the range of angles, but an ideal end result might be, for example, a TV that feels more like looking through a window instead of looking at a flat picture. As you walk side to side around the TV, you would be able to see new angles on the displayed image the same way you perceive new angles when walking past a window.
One of the side effects of this is that you can focus on near and distant objects in a light field display in exactly the same way you focus on near and distant objects in the real world. But the downside is that you need multiple times the resolution.
You need an entire full-resolution image for however many discrete viewing angles you support. We’re a long way from being able to provide the kind of video bandwidth that would be required to support a different image at the angle between two eyes at an average viewing distance for an entire 180-degree field of view. Such light field displays would require hundreds, possibly thousands, of times the video bandwidth TVs use today.
How different are Conventional VR Displays?
VR headsets like the Oculus Rift or the HTC Vive artificially create images that seem to have depth by using visual cues that include:
- Binocular Disparity or Stereopsis: Your right and left eyes each see a somewhat different image. When the brain processes these two images, the disparities give the illusion of depth.
- Motion Parallax: When you move your head side to side, whatever is closer to your sight seems to be laterally moving faster than objects located farther from your eyes. So VR headsets take advantage of this to fool your brain into believing that certain things in the scene are actually father than they are.
- Binocular Occlusions: Things that are in the foreground of a scene, and in front of other objects, appear to be nearer. This results in a simple relative ranking of distance. But if the occlusion is different for each eye, the human brain registers this as depth.
- Vergence: When human eyes look at something, the closer the object is to the eyes, the more the eyes have to converge to keep it in the center of the viewer’s field-of-view. But if something is far, the eyes have diverged until, at an infinite distance, the center of gaze for the eyes are parallel. This rotation gives the brain the data it needs to calculate the distance to the object.
Most HMDs don’t present any focus cues. The whole scene is permanently in focus because it’s on a flat screen located at the same distance from the user’s eyes. But the eyes are diverging and converging depending on the part of a changing stereoscopic image to which the attention is drawn to.
The brain does not feel comfortable with this, and this “vergence-accommodation conflict” results in “visual discomfort, compromised image quality, nausea, headaches, diploic vision, eyestrain, and fatigue.
Who is Working on This?
Nvidia and Stanford are working together in order to develop this new display technology. They call it light field stereoscope and it uses two-layered displays, with two LCD panels separated by 5mm, in a single headset. The VR headset uses a microlens array to split each image into individual light rays, tracking and displaying where each one has come from and the direction it is traveling. This makes it easier for human eyes to fix on focus cues at different depths.
Start up Magic Leap claims it is working with light fields for its Augmented Reality technology. The development promises to amaze us because they assure users’ eyes will have light from the real world to process in addition to virtual images.
Do you have something to say about light field displays? Share it with us in the comments.