Super-resolution imaging with diagonal sampling

The charge-coupled device (CCD) revolutionized photography by enabling the capture of light electronically, as recognized by the 2009 Nobel Prize in Physics. However, CCD/CMOS pixel size has become a bottleneck for digital imaging resolution.

The problem stems from a formal difference between the rectangular sensor and the circular or symmetrical lens. Peng Xi, associate professor of biomedical engineering at Peking University, explains, "In a lens-based imaging system, the lenses are mostly circular-symmetric, yet the CCD/CMOS sensors are all rectangular. This results a circular-symmetric transfer function in the optical system, and a rectangular data collection in the frequency domain."

Targeting that difference, an international research team led by Xi recently investigated the frequency domain sampling characteristics of CCD/CMOS imaging. Their research, reported in Advanced Photonics, found that higher frequency domain information can be obtained in the diagonal direction, when the optical transfer function is greater than the side length of the rectangle. Xi explains, "The Fourier transform of rectangular CCD data is still rectangular, so the diagonal direction can collect up to 1.4 times higher frequency than the horizontal or vertical direction." Based on this principle, the resolution can reach 1.5 pixels when samples are combined diagonally, denser than the conventional resolution of 2 pixels.

Frequency-domain extension microscopy

Guided by this insight, Xi's team proposed a novel technology: frequency-domain diagonal extension (FDDE) microscopy. To demonstrate, they established a frequency domain diagonal imaging platform, based on a lensless microscope with a complementary metal-oxide-semiconductor (CMOS) chip. Lens-free microscopy (LFM) breaks with conventional lens-based microscopic techniques by avoiding the application of the lens. Xi explains, "LFM is not limited by a lens system, and has the additional advantage of sufficiently large frequency components."

To enable lens-free imaging of a sample at different angles, a 2D detector is mounted on a manual rotating platform. A series of images is obtained at different detection directions and co-registered. The high-frequency information associated with the fine structures of the data obtained from different directions is then extracted, stitched together algorithmically, and converted back into the spatial domain to obtain a super-resolved image.

Rich biological structures visible

Biological samples often contain rich structures, ideal for testing the performance of FDDE. In one test, the team imaged a mouse skin sample, acquiring three rotational holographic raw images from different angles. The frequency domains of these three images were then synthesized through FDDE, revealing fine details not observable with a single holographic image, but clearly resolved via FDDE. In another test, the team imaged blood cell smears. The circular structure of most blood cells, which appears oddly rectangular in conventional LFM, was clearly distinguished as a ring shape using the FDDE technology.

FDDE imaging with a mouse skin sample. (a) The FDDE LFM image of the mouse skin sample. (b) An enlarged view of the region marked in (a). (c) LFM images. (c1), (c2), and (c3) are the same area as (c4) in the three-phase images with different orientations. The arrows in the upper-right corner correspond to the direction of the sample in the experiment. The three arrows indicate the FDDE image. In addition, (c2) and (c3) and (d2) and (d3) are rotated back to the same direction as in (c1) and (d1), respectively, for a comparison. The line profile in (c4) is marked between the arrows. The inset in (c4) is imaged with a 10× bright-field microscope, presented as the ground truth. (d) The frequency domains of the three-phase images and the FDDE image. The yellow rectangle is the boundary of the lens-free microscope. The red line area in (d1)-(d3) is combined into (d4) based on the principle of FDDE.

After demonstrating the FDDE's performance in lens-free microscopy, the team demonstrated that the principle of enriched resolution through diagonal sampling can be extended to lens-based photography, when resolution is limited by pixel size. Consistent with the principle of FDDE, they achieved a resolution 1.3 times higher diagonally than horizontally.

Columbus' egg?

Xi noted that FDDE is a "typical Columbus' egg-type problem" where a solution appears simple in retrospect: "The solution becomes very straightforward when looking at the difference between lens and CCD in the frequency domain." Xi anticipates that the method can be applied to many other areas where CCDs are employed, such as telescope imaging, machine vision, and spectroscopy.

Read the open access research article: S. Jiang et al. "Frequency-domain diagonal extension imaging," Adv. Photon. 2(3), 036005 (2020), doi 10.1117/1.AP.2.3.036005.