This paper introduces 3D HDTV relay broadcasting experiments of 2002 FIFA World Cup Korea/Japan using a terrestrial and satellite network. We have developed 3D HDTV cameras, 3D HDTV video multiplexer/demultiplexer, a 3D HDTV receiver, and a 3D HDTV OB van for field productions. By using a terrestrial and satellite network, we distributed a compressed 3D HDTV signal to predetermined demonstration venues which are approved by host broadcast services (HBS), KirchMedia, and FIFA. In this case, we transmitted a 40Mbps MPEG-2 transport stream (DVB-ASI) over a DS-3 network specified in ITU-T Rec. G.703. The video/audio compression formats are MPEG-2 main-profile, high-level and Dolby Digital AC-3 respectively. Then at venues, the recovered left and right images by the 3D HDTV receiver are displayed on a screen with polarized beam projectors.

Depth perception in images and video has been a relevant research issue for years, with the main focus on the basic idea of "stereoscopic" viewing. However, it is well known from the literature that stereovision is only one of the relevant depth cues and that motion parallax, as well as color, brightness and geometric appearance of video objects are at least of the same importance - with the individual influence being mainly dependent on the object distance. Thus, for depth perception it may sometimes be sufficient to watch pictures or movies on large screens with brilliant quality or to provide head-motion parallax viewing on conventional 2D displays. Based on this observation we introduce an open, flexible and modular immersive TV system that is backwards-compatible to today's 2D digital television and that is able to support a wide range of different 2D and 3D displays. The system is based on a three-stage concept and aims to add more and more depth cues at each additional layer.

In this paper, a new 3D object tracking system is suggested by using the disparity motion information extracted from input stereo image sequences. That is, in the proposed method, by using only the disparity information of input stereo image sequences, moving object can be detected and the location coordinates of the target are extracted and then, using these coordinate values the stereo object tracking is accomplished by just controlling the stereo camera that is mounted on a pan/tilt system. And at the same time, this disparity information can be sent to the receiver together with one of the input stereo image for multiview stereoscopic 3D image reconstruction of the target image under tracking. From some experiments with the 5 frames of the stereo image pairs having 256 x 256 pixels, it is shown that the proposed stereo tracking system can adaptively track the target object with a low error ratio of about 3.38 % on average between the detected and actual location coordinates of the target object.

In this paper, a new stereoscopic image reconstruction technique using an adaptive disparity estimation algorithm is proposed and its performance is analyzed in terms of PSNR through comparison to that of the conventional disparity estimation algorithms. In the proposed method, the feature-based disparity estimation method in which Canny mask operator is used for detecting the edge information from the input stereo pair are used for extracting the feature value. And, the matching window size for reconstruction of stereoscopic image is adaptively selected depending on the magnitude of the feature value of the input stereo pair by comparing with the predetermined threshold value. That is, coarse matching is carried out in the region having a small feature value while dense matching is carried out in the region having a large feature value. This new approach can not only reduce mismatching possibility of the disparity vector mostly happened in the conventional dense disparity estimation with a small matching window size, but also reduce the blocking effect occurred in the disparity estimation with a large matching window size. From some experimental results, it is found that the proposed algorithm improves PSNR of the reconstructed image about 5.36~7.76 dB on the average than that of the conventional algorithms.

In this paper we will present the concept of a modular three dimensional broadcast chain, that allows for an evolutionary introduction of depth perception into the context of 2D digital TV. The work is performed within the framework of the European Information Society Technologies (IST) project "Advanced Three-dimensional Television System Technologies" (ATTEST), bringing together the expertise of industries, research centers and universities to design a backwards-compatible, flexible and modular broadcast 3D-TV system. This three dimensional broadcast chain includes content creation, coding, transmission and display. Research in human 3D perception will be used to guide the development process. The goals of the project towards the optimized 3D broadcast chain comprise the development of a novel broadcast 3D camera, algorithms to convert existing 2D-video material into 3D, a 2D-compatible coding and transmission scheme for 3D-video using MPEG-2/4/7 technologies and the design of two new autostereoscopic displays.

The development of 3D TV systems and displays for public use require that several important criteria be satisfied. The criteria are that the perceived resolution is as good as existing 2D TV, the image must be in full natural colour, compatibility with current 2D systems in terms of frame rate and transmission data must be ensured, human-factors concerns must be satisfied and seamless autostereoscopic viewing provided. There are several candidate 3D technologies, for example stereoscopic multiview, holographic and integral imaging that endeavor to satisfy the technological and other conditions. The perceived advantages of integral imaging are that the 3D data can be captured by a single aperture camera, the display is a scaled 3D optical model, and in viewing accommodation and convergence are as in normal sighting (natural) thereby preventing possible eye strain. Consequently it appears to be ideal for prolonged human use. The technological factors that inhibited the possible use of integral imaging for TV display have been shown to be less intractable than at first thought. For example compression algorithms are available such that terrestrial bandwidth is perfectly suitable for transmission purposes. Real-time computer generation of integral images is feasible and the high-resolution LCD panels currently available are sufficient to enable high contrast and high quality image display.

We propose the use of synchronously moving micro-optics (lenslet arrays) for image pickup and display in three-dimensional integral imaging to overcome the upper resolution limit imposed by the Nyquist sampling theorem. With the proposed technique, we show experimentally that the viewing resolution can be improved without reducing the three-dimensional viewing aspect of the reconstructed image. Recording elemental images with enough resolution is also an important factor in determining the viewing resolution of the reconstructed image. We investigate the use of an optically addressed spatial light modulator, because it provides potentially better image resolution than the conventional CCD and liquid crystal display pair. We present experimental results using a liquid crystal light valve. A two-step integral imaging system to obtain three-dimensional orthoscopic real images is also presented.

We have developed a real-time color autostereoscopic display system using a reconstruction method of parallax rays. The notable point is to be able to display a volumetric image such as holography. Moreover, the data of this system can reduce much than that of hologram. Also, this system is visible to multiple viewers at the same time without special glasses. Well-known integral photography (IP) is similar to our system. Pseudoscopic images for IP are the most important problem that has to be solved. Using the method of ray-space representation on our display, it is not probability to appear pseudoscopic image. This paper describes the three-dimensional (3D) image input system based on ray-space representation for a color 3D display using a method of the reconstruction of parallax rays. The ray-space representation is one of image-based rendering techniques. It uses images of real objects to recreate photo realistic images of complex shaped objects without any explicit geometric model. Using similarities between a reference plane in ray-space representation and the screen of this 3D display, parallax rays for a 3D image, which we can observe at arbitrary viewing point, compose of amount of some parallax images. As an experimental result, the 3D image of origami bird (crane) is reconstructed.

Several types of auto-stereoscopic display systems have been developed. We also have developed a real-time color auto-stereoscopic display system using a reconstruction method of parallax rays. Our system consists of an optical element (such as lens array, a pinhole, a HOEs and so on), a spatial light modulator (SLM), and an image-processing unit. On our system, it is not probability to appear pseudoscopic image. The algorithm for solving this problem is processed in an image-processing unit. The resolution limitation of IP has studied by Hoshino, Burckhardt, and Okoshi. They designed the optimum width of the lens or the aperture. However, we cannot apply those theories to our system. Therefore, we consider not only the spatial frequency measured at the viewpoint but the performance of our system. In this paper, we describe an analysis of resolution for our system. The first we consider the spatial frequency along the depth and the horizontal direction respectively according to the geometrical optics and wave optics. The next we study the performance of our system. Especially, we esitmate the cross talk that the point sources from pixels on an SLM cause by considering to the geometrical optics and the wave optics.

We present experiments on an enhanced three-dimensional (3D) integral imaging system using double display devices, in which two 3D sub-images that cover different depth ranges are separately generated in each device and then they are combined using a beam splitter to reconstruct the whole 3D image with an enhanced depth of view. In a similar manner, the double-device system can also be used to obtain a wider viewing angle by combining two images with different viewing angle ranges. We discuss the possibility of three-dimensional integral imaging systems using multiple display devices, as extensions of the system with double display devices.

We have been developing a 3-D displaying method satisfying "Super Multi-View condition" that is the method of producing natural 3-D images by displaying multi-perspective images with fine perspective-pitch, which is narrower than the pupil diameter of viewer's eyes. To realize this method, we proposed a new 3-D display system constructed by a fan-like array of (many) projection optics (FAPO) and a concave mirror. The length of the system was very long (about 10 m). We tried to shorten the length using small size liquid crystal panels.

This paper proposes a new function of the two-dimensional lens array that is composed of many gradient-index lenses. The length of the lenses is an odd-integer multiple of the half period of the optical path. The array produces pseudoscopic three-dimensional (3D) images with reversed depth. Two lens arrays are positioned at a suitable distance so that orthoscopic 3D images with correct depth are formed in front of the lens arrays. The combined array captures, transmits and displays 3D images without other devices. A diffuser or opto-electronic amplifier can be inserted at the specific plane within the lens array.

A new and practical method for achieving color electro-holographic display systems was developed and tested with very promising results. In proposal method, virtual image reconstruction method and color hologram reconstruction techniques utilizing a single white light source, instead of the traditional three color laser lights, and three liquid crystal display panels (LCD) were proposed.. In this method imaging lens is not necessary and we can get wide viewing angle. A metal halide lamp with good specifications in each wavelength of the color components was adopted as the light source. Each of the three color light components required for the reconstruction of an image was extracted by color separation using dichroic mirrors. The separated color images were then reconstructed by the hologram formed on the LCD panels which is then combined togethre through the use of a prism to form a single 3D image. As the result system become compact and realistic one.

We present the initial results of a novel technique that uses only phase information from a digital hologram for the reconstruction of three-dimensional (3D) objects. Our holograms are created using phase-shift digital holography. Perspectives of the 3D object are usually reconstructed numerically on a computer. For large holograms this can be a computationally intensive task. We believe that the proposed reconstruction technique is promising for 3D display because the phase-encoded digital holograms admit optical, and therefore realtime, reconstructions that use commercially available display devices such as liquid crystal spatial light modulators. Numerical evaluation of the reconstructed 3D object and an experimental demonstration are presented.

In recent years there have been many activities in the area of three-dimensional display because of their potential applications. Large storage three-dimensional display system using integral imaging and volume holographic storage is proposed. In addition to the large storage, the speckle reduction based on time averaging and the viewing angle enhancement are achieved by using the proposed display system. Experimental results are presented and optical parameters of the proposed system are discussed in detail.

We introduced the following system and applied it to process of making holograms for science education. The procedures we proposed in this report are so simple that it is realized easily. By using this method, a hologram is produced simply and the process is safe and less expensive. Therefore, holography can be easily demonstrated to younger generation.

We investigated the effect of convergence of stereoscopic cameras on visual comfort and apparent depth. In Experiment 1, viewers rated comfort and depth of three stereoscopic sequences acquired with convergence distance set at 60, 120, 180, 240 cm, or infinity (i.e., parallel). Moderately converged conditions were rated either as comfortable (i.e., 240 cm) or more comfortable (i.e., 120 and 180 cm) than the parallel condition. The 60 cm condition was rated the least comfortable. Camera convergence had no effects on ratings of apparent depth. In Experiment 2, we used computer-generated stereoscopic still images to investigate the effects of convergence in the absence of lens distortions. Results matched those obtained in Experiment 1. In Experiment 3, we artificially introduced keystone distortions in stereoscopic still images. We found that increasing the amount of keystone distortion caused only a minimal decrease in visual comfort and apparent depth.

It is said that visual fatigue caused by watching stereoscopic images is due to the conflict between convergence eye movement and accommodation functions. We studied the degree of visual fatigue in subjects watching HDTV stereoscopic images. The HDTV stereoscopic images used as visual stimuli contained only absolute, with no relative, parallax. In the experiments, images were displayed behind or in front of the screen by a 120-Hz time-sequential method. The examination of visual fatigue was carried out with a five-grade subjective evaluation test and measurement of the accommodation response after watching for one hour. We found that when stereoscopic HDTV images were displayed within the corresponding range of depth of focus, and remained static in the depth direction, the degree of visual fatigue was almost the same as that induced by watching images displayed at the depth of the screen. However, when images were displayed outside the corresponding range of depth of focus, visual fatigue was clearly induced. Moreover, we found that even if images were displayed within the corresponding range of depth of focus, visual fatigue was induced if the images were moved in depth according to a step pulse function.

Two eyes' views together contain more spatial or geometric information than what can be represent faithfully from one vantage point. Just as it is impossible to paint an "authentic" picture on canvas, it is impossible for the visual system to make a "faithful visual" picture. We conducted several psychophysical experiments using stereoscopic displays. Experimental results on judging direction showed that the visual system solves the problem of fitting the two eyes' views, in their entirety, into a single cyclopean view by "displacing" and "compressing", perceptually, a portion of the visual field. The experimental results on judging shape showed that the visual system has a "correcting" mechanism to adjust for the consequences of the directional displacement and compression. We hypothesize that the operation of this "correcting" mechanism is triggered by the pictoral cue of occlusion and that this mechanism is responsible for the Kanizsa and Poggendorff illusions when three-dimensional stimuli are drawn a sheet of paper.

This paper describes a novel Free-Viewpoint TV system based on Ray-Space representation. This system consists of a multi-view camera system for 3-D data capturing, a PC cluster with 16 PCs for data processing such as data compression and view interpolation, input device to specify a viewpoint, and a conventional 2-D display to show an arbitrary viewpoint image. To generate an arbitrary viewpoint image, the Ray-Space method is used. First, the multi-view image is converted to the Ray-Space data. Then, interpolation of the Ray-Space using adaptive filter is applied. Finally, an arbitrary view image is generated from the interpolated dense Ray-Space. This paper also describes various compression methods, such as model-based compression, arbitrary-shaped DCT(Discrete Cosine Transform), VQ(Vector Quantization), and subband coding. Finally, a demonstration of a full real-time system from capturing to display is explained.

Laser radar (LADAR) is a device to acquire the profile of an object. A 3D LADAR range image is encoded as a 2D image whose gray values represent the measured range. It is possible to apply 2D filters for recognition of 2D encoded range image to detect the location of a target. However, if we want to detect the target coordinate within the input scene, that is not only the location of a target but also the distance from a sensor, 3D data processing is needed. First, it is needed to convert 2D range image back into 3D space, and then apply a 3D filtering technique to detect the target location and the distance from the sensor. We set up and solve the minimization problem that leads to the derivation of the 3D distortion tolerant nonlinear filter by minimizing the output energy due to the additive noise and the energy of correlation output in response to the input scene with the false objects. This filter also has a capability to reject a certain target if we know exactly what to reject. The correlation output at the distance level of the target shows a dominant peak at the target location, and the correlation outputs at the distance of the adjacent target level shows a very small peak at the target location. We can detect the location and the distance at the same time of the distorted target using proposed 3D optimum filtering.

There are several ways of processing Images from multiview cameras for full parallax image generation. Among them, the most used one is laying the images as a rectangular shape pixel cell on the image display panel. However, it cannot maximize the stereoscopic effect and viewing zone size in horizontal direction with a given full parallax image set. A diamond shape pixel cells can maximize the stereoscopic effect and the viewing zone size in vertical direction. The sequence of making the diamond shape pixel cell includes two rotations of a multiview image with the rectangular pixel cell in opposite to each other with the same angle.

In this paper, we propose a system for generating arbitrary viewpoint images. The system is based on image measurement and consists of three steps: HDTV image recording, modeling from images and displaying arbitrary viewpoint images. The model data is converted to VRML models. In order to estimate 3D shapes, we developed a new modeling algorithm using the block matching method as well as the volume intersection method. The proposed algorithm achieves fast and precise modeling. It is confirmed that the derived human model with motion can be smoothly played in a VRML browser on a PC, and the viewing position of the observer can be changed by mouse control.

The Argus project uses an array of computers and cameras as a means of investigating telepresence and real-time three-dimensional imaging. In this paper we will briefly discuss telepresence from an information flow and visualization perspective. The paper also includes a detailed description of the Argus hardware and a software layer developed to manage the imaging and computational resources. MPEG-2 and feature extraction will be described as parallel compression systems for the Argus camera array.

In this paper, we propose a method based on the Maximum Likelihood theory for removing the speckle pattern that plagues coherent images. The hypothesis on the speckle effect imply that the maximum likelihood criterion uses only intensity information. The model of the image is based on a lattice of knots corresponding to the vertices of triangles where the gray level of each pixel inside a cell is provided by linear interpolation.