Compound-eye-based multidimensional imager
Compound-eye optics enable a simple and scalable 2D scanner to observe more than 2D optical signals without sacrificing spatial or temporal resolution.
In imaging applications, such as security or biomedical imaging, increasing the number of dimensions of acquirable optical signals could help identify and classify objects in the image. However, conventional approaches to gain additional information, such as depth or spectral data, with a 2D image sensor, sacrifice spatial or temporal resolution. We have developed two promising approaches using compound-eye optics for future imaging systems that capture and make use of the additional dimensions without sacrificing resolution.
Our first approach is based on compressive sensing to observe the multidimensional signals. The optical design modulates each dimension, and a sparsity-based reconstruction solves the (ill-posed) inverse problem. The second approach uses dimensionally invariant optical design and signal processing to observe a 2D image of a 3D object with a wide field of view (FOV) and an extended depth of field (DOF).
The visual organs of insects and crustaceans are compound eyes. They are composed of lenslets, partitions, and detectors as shown in Figure 1. In the apposition type, each detector optically connects to a single lenslet. In the superposition type, a detector connects to multiple lenslets. Both types have unique features and are used in various innovative imaging systems, notably for compact imaging hardware.1
Optical signals have multiple dimensions, including 3D position (x, y, z), time (t), wavelength (λ), and polarization (p). Conventional approaches to observing these parameters with an image sensor compromise the lateral spatial resolution (x, y) or the temporal resolution. We have introduced an apposition compound eye for single-shot multidimensional imaging based on compressive sensing (CS).2 In this approach, the basic optical unit in the compound eye—made up of a lenslet, detector (sensor), and partition—is differently modulated in each physical dimension except the lateral spatial ones (x, y). Figure 2 shows schemes for multispectral imaging based on this approach. The object datacube (x, y, λ) is sheared as in Figure 2(a) or multiplied with spectral transparencies as in Figure 2(b) along the λ-axis. We use a sparsity-based algorithm to reconstruct the full-sized datacube.
We have applied this approach to depth imaging3 and multispectral polarimetric imaging.4 It is also applicable for wide-dynamic-range imaging and large-FOV imaging.5 We formerly proposed some CS-based methods for multidimensional imaging.6, 7 The approach mentioned here can reduce the size of the imaging hardware compared with our previous methods.
When an object is spread over a large 3D space (x, y, z), it is challenging to capture a 2D image of it. Wavefront coding is a well-known technique for increasing the DOF in an imaging system,8 and some wide-FOV imaging systems have been proposed with compound-eye optics.9 We have proposed two approaches to simultaneously increase the DOF and FOV using a compound eye without any additional optical element.
The first approach uses superposition compound-eye optics.10 Spherical aberration occurs by spherical arrangement of the basic optical units or distortion in them as shown in Figure 3.11 Spherical aberration increases the DOF by acting as a pseudo-depth-invariant point spread function (PSF).12Furthermore, the monocentric configuration of a spherically designed superposition compound eye increases the FOV. After capturing an intermediate image by the imaging optics with the 3D-invariant PSF, a sharp image with a large DOF and FOV is restored by deconvolution with the PSF. This method is also applicable for image projection,13 in which case a pre-deconvolved image is projected through the optics.
The second approach is based on apposition compound-eye optics,14 in which the compound eye acts as a microcamera array and captures the light field.15 This enables us to computationally realize an arbitrary camera condition including a wavefront-coded camera. We calculate an intermediate image supposing a wavefront-coded camera from the physically captured light field of a 3D object. A sharp object image is restored by deconvolution with a wavefront-coded PSF: see the experimental results in Figure 4. A single captured elemental image for a 3D object in Figure 4(a) has a large DOF but is noisy because of a high F-number. The intermediate image with wavefront coding in Figure 4(c) has a larger DOF compared with that without wavefront coding in Figure 4(b). The final deconvolution result with the wavefront coding, which has a large DOF, is sharp and noiseless: see Figure 4(d). This method is also applicable for image projection.14
In summary, we have presented several approaches to observing multidimensional optical signals based on compound eyes. They are promising foundations for next-generation imaging systems. Future work will focus on integrating each of them into compact imaging hardware. State-of-the-art manufacturing technologies, such as wafer fabrication, will be key to this.
Ryoichi Horisaki is an assistant professor, working mainly on computational imaging. He received a PhD from Osaka University in 2010.
Tomoya Nakamura is a PhD candidate whose research interests include computational imaging. He received an MS from Osaka University in 2012.
Jun Tanida is a professor and researches multi-aperture imaging and photonic DNA computation. He received a PhD from Osaka University in 1986.
