A novel approach to attaining high-resolution imaging
The image-resolution capabilities of digital sensors can be limited by several factors. One of these is the F number (i.e., the ratio of the focal length of the lens to the diameter of the aperture) or, equivalently, the quality of the optics, which is related to diffraction limitations. A second is driven by the signal-to-noise ratio and dynamic range, since only spatial information above the noise level may be detected. A third limitation related to the geometry of the digital-sensing array is the focus of this article. Figure 1 illustrates several of these factors. Figure 1(a) shows the original high-resolution image, while Figure 1(b) illustrates a diffraction-related limitation caused by the point-spread function (PSF), i.e., the response of an imaging system to a point source or object. In this case, the PSF is about ten times larger than the original image resolution. Figure 1(c) shows resolution reduction because of pixelization (a geometrical limitation). Finally, Figure 1(d) shows the loss of spatial features due to a small dynamic range or a low signal-to-noise ratio, which we simulated with a 1-bit binary sensor.
The geometrical-resolution limitation can be divided into two subtypes. The first involves the pitch of the sampling pixels, i.e., a combination of the distance between two adjacent sampling points and the total number of points. The second is related to the shape of each sampling point, or the spatial responsivity of each pixel. Ideal sampling is typically done using a mathematical train of delta functions. However, if the pixel size were a delta function, its energy efficiency would be very low. The common solution to the geometrical-resolution limitation related to the sampling-array pitch is commonly solved by microscanning,1 in which a set of images with relative subpixel shifts is captured. Improved geometrical resolution can be obtained by interlacing and applying digital interpolation. In Figure 2, the left- and right-hand images are the results after and before microscanning, respectively.
Unfortunately, the limitation of the spatial shape of each sampling pixel is more problematic than the pitch issue. We have taken three approaches to help solve this problem. In one, the spatial-binary mask (where the period equals the pitch of the pixels) was either attached to the detection array,2 placed in the intermediate-image plane, or projected onto the inspected sample3 (e.g., for microscopy-related applications). This mask modifies the Fourier transform of the PSF generated by the pixel shape. As a result, the Fourier transform of the modified PSF (after addition of the mask) will have no spectral zeros, so its effect can be removed easily by inverse filtering (i.e., division in the spatial-spectrum domain). The main benefit of such a design is that it generates an almost transparent mask that transmits most of the energy. In turn, the mask generates the required modification to the Fourier transform of the PSF, so that the modified distribution has values as far away from zero as possible. In this case, the inverse filtering is less sensitive to noise in the image.
The second idea4 takes advantage of the position of a spatial-binary mask in the intermediate-image plane. However, the mask is not periodic, and may even be random. The idea is not to modify or reshape the PSF of a single sampling pixel, but rather to represent the problem of the spatial-blurring function as a global operation over the entire image. This is achieved with the aid of linear algebra, by describing the operation as a vector multiplied by a matrix. The problem is that there are more variables than equations in such a representation. The purpose of including the random mask is to add equations and have an equal number of equations and variables so that the global-matrix-inversion operation can be realized with high mathematical stability and with low sensitivity to the noise in the image. A high-resolution image is obtained after inverting the blurring matrix. At the spatial positions where the binary mask blocks the light, we have a priori knowledge of the anticipated readout.
The matrix-inversion operation can be done only once, since it represents the system parameters (including the added mask) and is not image dependent. Figure 3 illustrates this approach. Figures 3(a), (b), and (c) show the original high-resolution image, the low-resolution image (without applying geometrical superresolution), and the superresolved reconstruction using the random mask of Figure 3(d).
Our third approach is related to aperture coding. We assumed that the diffraction limitation is much less restrictive than the geometrical problem. Thus, a special mask can be placed on the aperture of the imaging lens, performing orthogonal coding of different aperture regions. After downsampling due to the low geometrical resolution, aliasing occurs and various spectral regions are mixed together. However, by virtue of the presampling orthogonal encoding, they can still be separated, so that a high-resolution image may be reconstructed.5,6 We even used this approach to implement optical zooming to simultaneously obtain geometrical superresolution in the central area of the image as well as the original low resolution in its peripheral regions.7
Our ongoing work involves implementation of the second geometrical superresolving approach4 and microscopy-related setups for improved geometrical resolution and 3D capabilities.
Zeev Zalevsky acknowledges the significant contributions by his collaborators to different projects related to geometrical-superresolution techniques. Specifically, we are grateful for the contributions of David Mendlovic, Emanuel Marom, Dror Fixler, Javier Garcia, Aryeh Weiss, Moti Deutsch, Amikam Borkowski, Jonathan Solomon, and Alex Zlotnik.
Zeev Zalevsky is an associate professor in the School of Engineering. His major fields of research are optical superresolution, nanophotonics, and radio-frequency photonics. He has published two books, more than 10 book chapters and 180 refereed papers, and holds about 15 issued patents. In 2007 he received the Kril prize from the Wolf foundation, and in 2008 he was awarded the International Commission for Optics prize.
Bahram Javidi is board-of-trustees distinguished professor, which is the highest honor bestowed on a faculty member based on research, teaching, and service. He is a fellow of seven national and international professional scientific societies, including SPIE. He has received an extensive number of awards for his work, which includes more than 610 publications, 110 plenary and keynote addresses and invited conference papers, and 19 patents.
