Correct focus cues in stereoscopic displays improve 3D depth perception

Many different approaches to 3D displays have been explored, most of which are stereoscopic. These kinds of displays create depth perception by presenting two perspective images—one for each eye—of a 3D scene from two slightly different viewing positions. Stereoscopic displays have dominated the technology for a range of applications, including flight simulation, scientific visualization, medicine, engineering design, education and training, and entertainment systems. For instance, a see-through, head-mounted display merges virtual views with physical scenes, enabling a physician to see a 3D rendering of anatomical structures or computer-assisted tomography images superimposed on a patient's abdomen (see Figure 1).¹

Figure 1. Proof-of-concept demonstration of augmented reality in medical visualization.¹

Stereoscopic displays have evolved dramatically in recent years and, in fact, have driven the rapid revival of 3D cinema. By the same token, psychophysical studies have associated a number of visual artifacts with extended use of stereoscopic displays. Examples include apparent distortion in perceived depth, visual fatigue, double vision, and delayed and slowed accommodation responses or hyperopic changes (farsightedness).^2,3 One underlying cause may be a problem inherent in these displays known as accommodation-convergence discrepancy.

In conventional stereoscopic displays, pairs of images are typically presented on a 2D flat surface at a fixed distance from the viewer. The eyes are cued by the 2D images to accommodate to a fixed distance where the image planes are located. At the same time, the binocular disparity of the image pairs leads the eyes to converge towards different depths of the rendered 3D content. In other words, the natural coupling of eye accommodation and convergence in viewing a real-world scene is broken in stereoscopic displays. Moreover, unlike viewing a natural scene, in a simulated scene, retinal image blur does not vary with distance from a point of eye fixation to other points at different depths.

Varifocal- and multifocal-plane stereo displays represent a possible solution to the focus-cue problem, by either providing dynamic control of focal distance through active optics or presenting multiple focal planes at different distances. Varifocal-plane displays dynamically compensate the focal distance of a single-plane display based on a viewer's fixation point.⁴ Multifocal-plane displays concurrently present a stack of perspective images on multiple focal planes at discrete distances. Each focal plane is responsible for rendering 3D objects within a depth range centered on it. Multifocal planes are implemented either by spatially multiplexing a stack of 2D displays⁵ or by fast-switching the focal distance of a single 2D device.^6,7

We recently developed a prototype with addressable focus cues.^4,7,8 We used a liquid accommodation lens, which enables continuous addressable focus cues from optical infinity to as close as 8 diopters. The transmissive nature of liquid lenses allows a relatively compact and practical optical layout with no moving components. We demonstrated a bench prototype that permits a range of accommodation cues from 8 to 0 diopter. Our prototype flexibly controls the distance of a focal plane and the dioptric spacing between adjacent planes. It also operates the display in either varifocal-plane or time-multiplexed, multifocal-plane modes by changing the modes of the driving voltages applied to the liquid lens. Switching among various settings or modes does not require any hardware modifications. These unique capabilities enable flexible management of focus cues suited to a variety of applications requiring either wider depth range or better accuracy.

In varifocal-plane mode, the voltage applied to the liquid lens is dynamically adjusted through a user interface to render correct accommodation and retinal blur cues for objects at a specific depth. Objects at other depths will still have the mismatched accommodation and convergence cues. In multifocal-plane mode, the liquid lens is fast-switched between multiple driving voltages for objects of different depths. To create a flicker-free appearance of 3D objects rendered sequentially on multiple focal planes, the speed for the liquid lens, the microdisplays, and the graphics card is proportional to the number of focal planes.

Figure 2 shows experimental results of a dual-focal-plane configuration. We systematically designed a depth-fused display that can create a large viewing volume from 0D to 3D with six carefully placed planes and depth-weighted fusing functions.⁸ Figure 3 shows a simulated image, where the 3D object extends from 3 to 0.5 diopter and the eye is accommodated at a distance of 0.5 diopter.

Figure 2. Demonstration of a dual-focal-plane configuration. Photographs taken with the camera focused at (a) 16cm and (b) 1m.⁷

Figure 3. Simulated retinal image of a depth-fused, six-focal-plane display.⁸

We assessed accuracy of depth perception using a controlled depth-judgment task and measured a user's accommodative response with the prototype operating in monocular, varifocal-plane mode.⁴ The results suggest that displays with appropriately rendered accommodation cues indeed provide accurate perception and that the responses of the eyes match the depths of the display prototype. These technologies could have a significant impact on both application and usability of near-eye displays. In years to come, with the increasing popularity of 3D technologies, more research will be directed towards issues of usability and the social impact of stereoscopic displays. We are currently developing a depth-fused, six-focal-plane display prototype and planning a range of user studies to evaluate the effects of focus cues on both accuracy of depth perception and visual fatigue when using stereoscopic displays.

This work is supported by the National Science Foundation (contracts 06-44446 and 0915035).