Progress in European 3DTV research
Creation of ghost-like images of 3D objects and environments is the ultimate goal in video sciences. True 3D viewing will be far more exciting and satisfying than today's common 2D displays. The capture of 3D scenes, their abstract representation and storage, transmission, and finally their display are the primary technical components of 3DTV.
The history of stereoscopic 3D imaging is almost as old as that of 2D. Within a year or so after they invented conventional TV in the 1920s, engineers also demonstrated the feasibility of stereoscopic 3DTV. By the 1950s there were plenty of stereoscopic 3D motion pictures. However, although 2DTV flourished—it improved technologically, became commercially successful, and was eventually used widely—3DTV almost disappeared. The main reason for this failure is probably the fact that many viewers experience a motion sickness-like feeling, brought on as a consequence of a mismatch between perceived 3D cues. Recent technological developments have reduced the causes of viewer discomfort (also called eye strain) and, in 2004, a group of 19 European institutions from seven countries formed a consortium with the aim of solving many of the other technical challenges and bring 3DTV a step closer to reality.1,2
It is expected that stereoscopic 3D movies will become common and easily accessible after the conversion to digital cinema is completed in about three years: an increase in stereoscopic 3DTV broadcasts will follow. As with their historical counterparts, this near-future 3D viewing technology will need stereoscopic eye-wear. However, stereoscopic 3DTV is just the first step: although it is rather easy to achieve stereoscopy, it is a low-end technology for 3D viewing. Most of the current research activities in 3DTV displays are targeted at autostereoscopic viewing that does not require eye-wear. Better parallax is attained by presenting multiple stereo pairs, so that a feeling of seeing around the object can be achieved as the viewer changes his/her location, at least within a certain viewing angle. This will also allow many viewers to see the same 3D object at different angles depending on their positions. Head-tracking stereo displays are also under investigation by members of the research consortium.
‘Free view’ and ‘multi-view’ are commonly used terms to describe autostereoscopic displays. High-end 3D viewing involves true-parallax images and a viewer interaction as depicted in Figure 1. This is possible with different variants of holographic displays. A form of such incoherent viewing may be based on integral imaging concepts that have been known for about a century. Dynamic holographic displays for satisfactory holographic motion pictures are still far away: another decade might be needed before they become a commercial reality. However, basic research to investigate these high-end 3D displays is moving forward with considerable momentum.
The research consortium has developed and tested several different autostereoscopic displays and has also experimented with holographic techniques using spatial light modulators. Researchers have also studied various theoretical issues concerning digital processing and the generation of holographic and other diffraction-related signals.3,4 Such techniques are continuously being refined and we plan to integrate these across the different technological components to achieve end-to-end 3DTV operation.
Even though the user interaction is primarily with the display side of any TV system including 3DTV, shooting, storing and broadcasting content are equally important and demanding. There are various candidate technologies for 3D counterparts of these functional units: multi-camera solutions for capture; computer graphics methods for representation and storage; and adapted versions of digital compression and transport techniques for transmission are all under investigation with good progress.
Multi-camera capture of 3D environments require simultaneous recording using conventional cameras, but images are taken from many different angles. The cameras require careful calibration and synchronization, but with a sufficient number, 360° coverage can be recorded.
The consortium has already developed single- and multi-camera techniques for capturing 3D video. Many other experimental multi-camera systems have also been developed and 3D reconstruction of objects from their multi-camera views has been accomplished.
Digital video processing techniques have been developed to provide the view corresponding to an arbitrary angle (so called ‘virtual camera’) by interpolating views from the existing cameras. Such techniques are blended with modern computer graphics techniques, like wire-frames, to build a complete, but abstract, motion 3D content that can then be stored in computerized environments. To eliminate redundancy in such content, compression techniques that target high gains in storage and transmission by sacrificing only a little quality are being investigated. Streaming techniques for stereo and multi-view video are also in place.
Transmission of digital video over packet networks using common internet protocols is another challenging task: delays in synchronous frames from parallel video recordings and error concealment tasks are different from their 2D counterparts. Finally, it is our aim to deliver the same 3D content simultaneously to 3D monitors with different technical features. The goal is to provide service to both low- and high-end 3D monitors, as well as to current 2D monitors that can select their viewing angle and zoom interactively. It is envisaged, therefore, that the captured data will not be directly transmitted to the viewers, but the viewers will interact and fetch 3D data from a processed intermediate format.
The consortium is the largest of its kind in the world of 3DTV. It is the largest both in terms of number of contributing researchers and in terms of its technical scope. Successful commercialization in broadcast technologies depends heavily on standardization and, through some of our partners, we are one of the main contributors to MPEG4-3DAV standardization activities. Initial commercialization attempts will be based on stereoscopic and autostereoscopic displays, and the essential technical components are all already in place. However, commercialization of holographic, true-parallax systems—as depicted in Figure 1—will require another ten years or so.
One of the consortium's targets is a durable integration of researchers. We therefore expect to continue our collective efforts well beyond the life-time of this project. We expect many spin-off projects to continue, using the current collaboration as a platform, after this project ends.
