Current 3D technology is primarily focused on entertainment, such as movies and games, but other applications may also require 3D vision. For example, lower-dimensional displays cannot provide enough visual information for real-time interaction with 3D space, such as robot manipulator tele-operation, vehicle remote-control, or virtual simulators of these tasks. To effect direct manipulation with 3D vision, the display should provide not only a sense of depth, but also precise 3D positions of objects in the scene. Most conventional stereoscopic displays provide binocular parallax, which is sufficient to enhance the sense of depth for entertainment purposes. However, for precise depth perception, the display should also provide focal depth, motion parallax in horizontal and vertical directions, and correct any undistorted image geometry.
Our work involves coarse integral volumetric imaging (CIVI), a 3D display approach that provides not only binocular parallax, but also motion parallax and focal depth without geometrical distortion.1,2 CIVI combines volumetric and multiview techniques based on integral imaging (see Figure 1). Combining a fly-eye lens sheet (fine convex lens array) with a high-resolution display panel, integral imaging is a prominent 3D display system in the sense that it features both horizontal and vertical parallax. Conventionally, the number of pixels covered by each fly-eye lens sheet elemental lens is usually the same as the number of views, which means that the viewer perceives each elemental lens as one pixel. Therefore, the viewer's focus is always fixed on the screen (fly-eye lens sheet), which creates difficulties in displaying realistic 3D images that appear to emerge from the screen. CIVI uses a coarse convex lens array with an elemental lens large enough to cover pixels dozens of times greater than the number of views.1,2 Thus, many pixels of each elemental image are seen at the same time, and focal accommodation can be induced off-screen when each elemental image's real image is generated in the air.
Figure 1. Coarse integral volumetric imaging optical design.
When each elemental image is presented by multilayered display panels, we generate multilayered real-image planes in the air. By depicting each pixel on the image plane of the corresponding depth, we attain a natural focal depth effect.1,2 We realize smooth expression of 3D pixels between two image planes by depicting them on two adjacent panels, such that intensity is inversely proportional to the distance between the 3D pixels and the image planes.
The real image generated with a CIVI lens system includes various optical distortions and aberrations, which become more prominent with a wider viewing angle. CIVI corrects these optical distortions—including field curvature, barrel distortion, and chromatic aberration—with software. Each elemental image on the layered panel is distorted in the inverse direction, such that the observed image possesses correct geometry.
To realize a real-time interaction system with CIVI, we calculate and correct optical distortion on a real-time basis. Since field curvature, barrel distortion, and chromatic aberration in each elemental image is different from one another, we independently calculate and correct these optical distortions for each elemental image. With recent progress in graphics processing units, all of these distortions can be corrected on a real-time basis with shader techniques.3 Direct manipulation of virtual objects is now possible with CIVI technology. Based on our work so far, we are constructing a 3D camera system for CIVI, and we expect to use it for tele-operation.
University of Tsukuba
Hideki Kakeya received his PhD in engineering from the University of Tokyo in 1998. He worked for Communications Research Laboratory from 1998 to 2001. He is currently an associate professor.
1. H. Kakeya, Improving image quality of coarse integral volumetric display, Proc. SPIE
7237, pp. 723726, 2009. doi:10.1117/12.805469
2. H. Kakeya, T. Kurokawa, Y. Mano, Electronic realization of coarse integral volumetric imaging with wide viewing angle, Proc. SPIE
7524, pp. 752411, 2010. doi:10.1117/12.837309
3. S. Sawada, H. Kakeya, Coarse integral volumetric imaging with flat screen and wide viewing angle, Proc. SPIE
7863, pp. 78631L, 2011. doi:10.1117/12.872287