We have implemented and experimented several variations of 360 degree viewable tabletop color holographic display system. Variation of the implementation includes applying spatial tiling of SLM modules(no tiling, 1x2 or 2x2 tiling for mono or color displays), using different SLMs(DMD of different pixel pitch and resolution), and applying different structure of 360-degree floating image forming optics such as (1)double parabolic mirrors, (2)one parabolic mirror and one beam splitter, (3)one toroidal mirror, two spherical and one flat mirror. In this paper, we report results of various display system implementations based on several combinations of above-mentioned design options, with performance comparisons.

We study an integral three-dimensional (3D) TV based on the integral photography method, with the goal of developing a new form of broadcasting that provides a strong sense of presence. The integral 3D TV can display natural 3D images that have motion parallax in horizontal and vertical directions. To improve the image quality, we have developed an integral 3D TV system using multiple cameras and high-resolution display devices. This paper reports our latest research results of the development of integral 3D TV systems.

SLM (Spatial Light Modulator) with ultra fine pixel pitch (circa 1 micron meter) has been thought as a big issue to realize electronic hologram with wide viewing angle. SLM with LC light modulator controlled by TFT based backplane constructed on glass substrate is proposed according to the methods of scaling down flat-panel display technology. Introduction of sub micron meter patterning processes, the SLM panel with 3 micron meter pitch pixel was successfully developed for the first time. PCM (phase change material) has been used for memory devices and information recording devices. We propose a SLM with PCM such that information is recorded using electrical signal and read using light signal.

FTV (Free viewpoint Television) enables users to view a 3D scene by freely changing the viewpoint. It was developed based on ray-space representation. Omnidirectional FTV is 360-degree video with free viewpoint function. Ray capture and view synthesis are analyzed in the spherical ray-space and verified experimentally. Omnidirectional views with full parallax are successfully generated.

In this study, we examined a strategy where the images were reconstructed by CS in two steps. A first step with a small number of measurements to estimate the number of points needed, and a second step for the final reconstruction. In addition, we investigated the benefits to create a block- partition of the image to estimate locally the number of measurements needed for the reconstruction. We demonstrated that our strategy could be used to reconstruct images presenting a PSNR similar to the one obtained with the conventional method, but with fewer measurements.

We present a novel method to use an integral display not as a 3D display but as a device intended to improve visual accuity in subjects with some visual defect. We will provide the mathemathical basis behind this method. We propose to use a characterization of the response of every lenslet and every pixel and the estimated response in a human eye with some visual defect. Finally, an inverse system can be solved in order to produce a sharp image of the desired object, tailored for an specific subject with some visual defect.

In this paper, we propose a visual quality enhancement of 3D reconstruction algorithm in integral imaging. Conventional integral imaging has a critical problem that attenuates the visual quality of 3D objects when low resolution elemental images are used. Although, PERT is the one of solutions, the size of 3D scenes is different from optical reconstruction since it is not considering space between back-projected pixels on reconstruction planes. Therefore, we consider this space and use convolution operator. Especially, convolution operator can be designed by considering aperture shapes. To support our proposed method, we carry out optical experiment and computer simulations.

Integral microscopy is a technique that has the capability of capturing both, the spatial and also the angular information of a three-dimensional microscopic sample. The acquisition of this 3D information allows to computationally reconstruct the scene in different depths and also generate a set of perspective views. The main drawback of the integral imaging technology is the poor spatial resolution of the reconstructed images, that is limited by the diameter of the microlenses of the capture system. We have explored the limits of the geometrical model that sustains the integral imaging microscopy and also have developed an experiment in order to determine the optimal size of these microlenses in an integral microscope. We have concluded that we can work with microlenses whose diameter is 1/2 the size of the diffraction spot at the MLA plane.

Optical methods have been applied to visualize sound waves, and these have received a considerable amount of attention in both optical and acoustical communities. We have researched optical methods for sound imaging including laser Doppler vibrometry and Schlieren method. More recently, parallel phase-shifting interferometry with a high-speed polarization camera has been used, and it can take a slow-motion video of sound waves in the audible range. This presentation briefly reviews the recent progress in optical imaging of sound in air and introduces the applications including acoustic transducer testing and investigation of acoustic phenomena.

In this paper, we propose an optical three-dimensional (3D) visualization under inclement weather conditions. These conditions include fog and night environments. For visualization under fog, we use peplography technique which detects ballistic photons from the scattering media by photon counting imaging. In addition, we use photon counting imaging with Bayesian estimation and adaptive statistical parameters for night vision. To obtain 3D information under inclement weather conditions, we use integral imaging and computational reconstruction algorithm. Finally, we optimize these algorithms for real-time process and wearable devices.

In this research, we proposed a new cloud based medical data rendering and information management system. In this system, cloud computing is used for data storage and synchronization, Apache HTTP Server and MySQL are employed for database management and information distribution, and WebGL is utilized for real-time graphics rendering. The major contributions include that cloud data synchronization can effectively work with Apache HTTP Server for information real-time communication and data rendering management in a web browser, and real-time data visualization and medical diagnosis can be linked to database management through internet, providing users with intelligent feedback and data organization.

Traditionally, an imager only exploits the line-of-sight optical signal to infer information about the three-dimensional scene. However, the non-line-of-sight optical signal generated by interaction of ambient illumination with objects outside camera’s field of view (FoV) via the reflection and/or scattering processes, encodes information about the extended scene i.e. beyond the FoV. Using an information-theoretic analysis, based on a physics-based forward model, we quantify the fundamental limits of three-dimensional imaging and sensing of objects outside the FoV or line-of-sight of an imager using lightfield measurements. Our theoretical analysis and experimental results demonstrate that one can estimate object position information and achieve low-resolution imaging with scattered light (outside the FoV) using only passive lightfield measurements.

The integral photography was generated from a multi-view stereoscopic image. The fixation point is set in the capturing of the multi-view stereoscopic image on a computer. Consequently, there is a possibility that the effect of setting the fixation point, which is comparable to the one of the toed-in camera configuration in the capturing the stereoscopic image, influences the integral photography image. The distortion of the re-produced image was examined computationally at the depth and width. As a result, it was found that the linearity of the depth and width are maintained when the object is located around the fixation.

Plenoptic imaging techniques were employed to overcome the difficulties in object tracking. Our approach for object tracking takes advantage of the capabilities that plenoptic imaging provides such as refocus, perspective shift, and depth calculation to improve tracking accuracy. Our approach firstly uses depth information of the scene to distinguish the object from the background Secondly, it takes advantage of refocusing and perspective shift capabilities to cope with appearance variation problem. The experimental results showed that our novel approach drastically improved the tracking accuracy especially for the image sequences containing objects rotational movements and partial occlusion as well as the cluttered background.

This paper proposes a new way of forming an aerial three-dimensional (3D) image that gives viewers smooth motion parallax. Our proposed system forms the real image of an arc 3D display in the mid-air with aerial imaging by retro-reflection (AIRR). The arc 3D display consists of arc-shaped scratches and gives a smooth motion parallax based on directional scattering. Optics setups to extract the scattered light for the aerial 3D display has been designed and realized with a prototype aerial 3D display. Our proposed display forms a 3D image in the mid-air by use of a single LED illumination.

3D Sensors have really matured in the recent years and is starting to play an interesting role in forensic science. One of the challenges within crime scene investigation is to be able to understand and make other people understand how a series of events occurred and what the evidence actually shows. By using 3D it becomes possible for everybody to visit the crime scene and gain a better and more accurate understanding, it also enables new possibilities for remote collaboration and training.

The knowledge of the exact complex (amplitude and phase) wavefield scattered for different illuminating beams allows the computation for example of the 3D spatial distribution of a specimen.  This work looks on how diffraction tomography behaves under high numerical aperture focusing condition.   Scalar theory is no longer valid in such system and non-paraxial vectorial field theory must be used.  'Numerical methods such as FDTD techniques can also be used to investigate the interaction of this field with the specimen.

Three kinds of phase modulation for three-dimensional imaging are introduced. One is phase modulation for wave-splitting generalized phase-shifting digital holography. Owing to the designed phase modulation, phase-shifting errors can be reduced. Other is phase modulation for twin image reduction of in-line digital holography. Owing to the designed phase modulation, the overlapping an object image and its conjugate image can be reduced. The last is phase modulation for computational ghost imaging. Owing to the designed phase modulation, the number of measurements to obtain a reconstructed image can be reduced.

In this invited presentation, we will introduce our recent research on 3D imaging based on common-path off-axis incoherent digital holography. In the proposed technique, two focusing lenses with different diffraction gratings are implemented by a phase-mode spatial light modulator and then two diffracted incoherent light waves can interfere at the image sensor. This system can be compact and be stable. We will present several experiments including LED sources. We will discuss the potential of the biological imaging.

Two separate experiments examined user performance and viewer discomfort during virtual precision alignment tasks while viewing a stereoscopic 3D (S3D) display. In both experiments, virtual camera separation was manipulated to correspond to no stereopsis cues (zero separation), several levels of microstereopsis (20, 40, 60, and 80%), and orthostereopsis (100% of interpupillary distance). The results support the use of orthostereopsis for critical, high-precision manual spatial tasks performed via stereoscopic 3D display systems, including remote surgery, robotic interaction with dangerous or hazardous materials, and related teleoperative spatial tasks.

In Non-Line-of-Sight (NLOS) scenarios the target is hidden from both the transmitter and the receiver by an occluder, i.e. a wall. Recent advancements in technology, computer vision and inverse light transport theory have shown that it is possible to recover an image of a hidden target by exploiting the temporal information encoded in multiple-scattered photons. The core idea is to acquire data using an optical system, composed of an ultra-fast laser that emits short pulses and a camera capable of recovering the photons time-of-flight information. We reconstruct 3D images from this data based on the backprojection algorithm, a method typically found in the computational tomography field, which is parallelizable and memory efficient, although it only provides an approximate solution. Here we present improved backprojection algorithms for applications to large scale scenes, such as NLOS imaging of rooms and lunar caves.

Fusing 3-D generated scenes from multiple, spatially distributed sensors produces a higher quality data product with fewer shadows or islands in the data. The implementation of this fused system allows simultaneous real-time streaming of point clouds from multiple assets, tracking of personnel and assets in that fused 3D space, and visualization on a Microsoft HoloLens mixed-reality device. Challenges that were: 1) the tracking and synchronization of multiple independent assets; 2) identification of the network throughput for large data sets; 3) the coordinate transformation of collected point cloud data to a common reference; and 4) the fused representation of collected data.

Three dimensional (3D) fluorescent images of optically thick samples exhibit spherical aberration (SA) due to the refractive index mismatch between the imaging layers. Space-invariant (SI) restoration methods (a.k.a. deconvolution) result in restoration artifacts because they do not account for the SA in the data. To address the impact of SA in 3D microscopy we have been developing: 1) computational approaches based on imaging models that account for SA; and 2) a computational-optical approach that engineers the system’s response to be less variant with SA, thereby enabling the use of SI restoration. Simulated and experimental results that evaluate our different approaches will be shown.

Foam cells, particle conglomerates, biological tissue slices and colloidal suspensions are just a few examples of collections that create an image with multiple touching or overlapping regions. Finding the diameter statistics of regions with incomplete or overlapping perimeters is tedious and computationally intensive, especially for large 3D data sets. This paper provides the mathematical foundation for a newly discovered relationship that allows for a simple and accurate characterization of the aggregate diameter associated with blob collections.

Light fields are 4D parametrizations of rays extensively used for the representation of the rays emitted from 3D objects. With the increasing availability of spectral imagers, the conventional light fields can be augmented by the spectral information, yielding to a 5D ray parametrization referred to as spectral light fields (SPL). In this paper we show the utility of the spectral light fields for digital 3D reconstruction. We demonstrate the utilization of the spectral light fields for profilomety and refocusing.

We developed a high-resolution integral imaging display method with multiple ultra-high definition projectors. In the proposed method, elemental images are projected one-over-another onto a lens array using projectors placed at optimal positions. By projecting elemental images at predetermined angles as collimated light rays, the resolution and viewing angle of the integral image can be enhanced. We prototyped a display system consisting of six projectors and realized the reproduction of an integral image with a resolution of 100 thousand pixels and a viewing angle of 30 degrees. Currently, we are improving our method of reproducing light rays while refining our display system configuration to achieve near-standard television resolution. Latest research results are also presented.

Reconstructing 3D models is playing an important role in many applications. Instead of employing a collection of cameras and/or sensors, this paper proposes a simple way to build a cheaper system for 3D reconstruction using only one depth camera and two mirrors. Each mirror is equivalently considered as a depth camera at another viewpoint. Since all scene data are provided by only one depth sensor, our approach can be applied to moving objects and does not require any synchronization protocol as with a set of cameras. Experiments were performed on different objects to confirm reconstruction accuracy of our system.

For the visualization of naturally perceivable 3D scenes with continuous range of observation angles in dedicated displays with multi-layer diffuser 3D screens (also called - volumetric 3D displays) development of novel 3D image data rendering approaches is required. The new rendering methods shall take into consideration a requirement for a real-time interaction with the 3D image by the observer. In this work, several key 3D image rendering algorithms are discussed: multi-layer anti-aliasing (for different scenarios - 3D object and volumetric 3D image data sets), visible surface enhancement, linear and non-linear transformations, such as translation, rotation, scaling, colour and intensity mapping.

We overview a recently published technique for object visualization and detection in low illumination conditions and in the visible spectrum using passive three-dimensional (3D) integral imaging (II). The method overviewed does not require photon-counting or cooled CCD cameras. Moreover, two-dimensional imaging systems that operate in the visible spectrum may not perform well in photon-starved conditions. The 3D imaging system overviewed is implemented by first recording multiple two-dimensional (2D) elemental images (EIs) from the photon-starved scene containing an object behind occlusion. After image acquisition, a reconstructed 3D integral image is computed allowing for object visualization. Automatic object detection is then performed.

In this paper, we propose a new high-resolution depth estimation algorithm in integral imaging which can obtain three-dimensional (3D) images by using lenslet array. In conventional studies, a stereo-matching is used for depth estimation. However, it is not the best solution for integral imaging since the 3D images are usually low-resolution images. Therefore, we propose a pixel blink rate based algorithm using pixel of the elemental images rearrangement technique (PERT) in integral imaging. Through our optical experiment, the depth resolution by our technique is 100 times higher than the one by conventional method.

In this paper, we propose a new 3D image sensing and visualization technique of integral imaging to improve the lateral and depth resolutions. We suppose that the 2D elemental images are important information for the 3D performance of integral Imaging because each images include lateral and depth resolutions of 3D objects. Therefore, we analyze the basic parameters of camera array or lenslet array for the use of the resolution priority integral imaging (RPII) and the depth priority integral imaging (DPII). To show the feasibility of our proposed method, we implement the optical experiments and computer simulations.

In this paper, we propose a new computational reconstruction technique of integral imaging for depth resolution enhancement. In conventional integral imaging, we can record and visualize 3D object by a lenslet array. However, due to low resolution of elemental images, the reconstructed 3D image has low visual quality and depth resolution. To solve this problem, we propose the computational reconstruction technique using continuously non-uniform shifting pixels instead of integer uniform shifting pixels in conventional computational volumetric reconstruction. Our experimental results show the improvement of depth resolution and visual quality of the reconstructed 3D image.

In this work, we present frequency estimation system based on digital holography for vibrating objects such as piezoelectric vibrator. The proposed method employed on off-axis digital holography geometry where an object wave passes perpendicular and near to vibrating objects by superimposing with a reference wave. The interference patterns are recorded as digital holograms by using a high speed image sensor as function of time. Finally, to estimate vibration frequency of object, Fresnel reconstruction of free space propagation and some data processing techniques have been used. We present experimental results in support of proposed frequency estimation system for piezoelectric vibrator.

We overview a previously reported method for three-dimensional (3D) profilometric reconstruction with occlusion removal based on flexible sensing integral imaging. With flexible sensing, the field-of-view of the image system can be increased by randomly distributing a camera array on a non-planar surface. The camera matrices are estimated using the captured multi-perspective elemental images, and the estimated matrices are used for 3D reconstruction. Object recognition is then implemented on the reconstructed image by nonlinear correlation to detect the 3D position of the object. Finally, an algorithm is proposed to visualize the 3D profile of the object with occlusion removal.