Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XXXII

A flat glass is presented to enhance the image resolution.The light is shifted to half a pixel by adjusting the rotation angle of the flat glass.

Multispectral imaging has gained traction to create richer datasets for machine learning driven analysis of samples such as cell culture and stained tissue. We propose a system that can scan across spatial frequencies to capture a variable resolution multispectral datacube to increase imaging throughput. We simulate such a system by adapting Fourier ptychography (a illumination-coded microscopy technique that builds up the Fourier space of an object with more acquisition) using a spectrally coded aperture. Such a system could increase throughput by minimizing the acquisition time required for inference on each sample.

In Stimulated Raman Scattering (SRS) Microscopy, the lock-In amplitude and phase provide complementary information that is easily measured in any experiment. This approach characterizes both the strength of a signal as well as the lifetime of the underlying process on a per-pixel basis. In this work, we use spectral focusing SRS Microscopy to characterize a lithium ore sample. Through leveraging spectral focusing SRS and all the information from the lock-in amplifier, we show that it is possible to fully characterize the overlapping nonlinear optical processes occurring in a lithium ore sample by performing a full scan of the pump-probe cross-correlation. We also show that by explicitly leveraging the correlations between the phase and amplitude, it is possible to enhance the quality of segmentations and the product of preprocessing techniques in the form of bilateral filtering.

Deep ultraviolet (UV) microscopy is a high-resolution, molecular imaging technique that typically constructs 2D images of the axial projection of a sample’s 3D absorption. Here we present a multispectral tomographic imaging approach to visualize complex 3D structural features in samples using partially-coherent, asymmetric illumination patterns. We aim to use through-focus intensity image stacks to extract 3D absorption and refractive index (RI) distributions of the sample at distinct UV wavelengths and to simulate the optical transfer function via a solution to the inverse scattering problem to close agreement with experimental data.

A typical configuration is a two-piece assembly consisting of a long-pitch GRIN lens followed by a short-pitch lens. The GRIN image relay eliminates the pixilation problem of fiber bundles/MCFs and the instability of MMFs due to bend-induced mode mixing. Using a multi-pitch GRIN lens with a high numerical aperture. However, the cascading of the multiple pitches leads to accumulation of aberrations. We show that these aberrations can be compensated at the proximal end using predistortion; that is, projecting the phase conjugate of the point spread function (PSF) instead of a Gaussian beam into the multi-pitch GRIN lens for applications, such as fluorescence or photoacoustic imaging. Real samples with high-NA and 6 to12 multi-pitch GRIN lenses of diameters 0.2, 0.5, 0.7 and 1.0 mm have been evaluated for this application.

Fourier light-field microscopy (FLFM) enables scanning-free, rapid, volumetric imaging using an MLA to simultaneously record spatial and angular information. However, current FLFM realize multicolor imaging with a switchable illumination, preventing them from simultaneously sensing biological signals on multiple channels. Here, we introduce instant two-color FLFM (IDC-FLFM). By equipping a filter array behind the MLA, IDC-FLFM allocates information among spatial, angular, and chromatic dimensions, capable of volumetric acquisition simultaneously in two channels at camera-limited speed, with single-cell resolution across an imaging field approaching millimeters and depths over hundreds of micrometers, facilitating the multiplexing observation of fast biological events. We characterized IDC-FLFM multidimensionally with various phantom samples, and demonstrated its imaging capacity on diverse biological specimens, for time-lapsed 3D recordings of quick morphological change and functional traits, proving IDC-FLFM as a promising tool for potential research in pharmacy, genetics, cardiovascular development, and cognitive neurology.

In biology, fluorescence microscopy has been widely explored for imaging due to the possibility of new contrasts, contrast enhancements, specific molecule detections, etc. Moreover, due to our eyes and cameras detection range, most of the fluorescence microscopy works in the visible. However, it is interesting to see other wavelengths where no multi-array cameras are available and only single-pixel detections are possible like one, for example, by laser beam scanning methods where a focal point produced by a spherical lens is scanned to generate an image point-by-point. In order to improve the image acquisition speed, it is possible to use a focal line produced by a cylindrical lens to reduce the scanning along one axis perpendicular to the laser beam line. In this context, here we present an focal line imaging method based on the signal derivative. Specifically, we obtain a fluorescent sample’s image line using a simple approach we call the derivative optical imaging technique (DOIT). As a proof of principle, we did fluorescent images of stained and unstained onion cells, for instance, using a low-power CW 532 nm laser. Image acquisition time and resolution are discussed.

We present an approach for simultaneous bright field (BF) and laser-based quantitative phase imaging (QPI) in which a fiber optic digital holographic off-axis Mach-Zehnder interferometer setup with a single digital camera sensor is applied. Recording of grayscale multiplexed images enables high-speed multimodal BF and QPI imaging that is only limited by the frame rate of the acquisition sensor. BF and QPI images are recovered from the recorded multiplexed images by Fourier filtering in combination with weighted image subtraction. The performance of our approach and its application in imaging flow cytometry are demonstrated by experimental results from the system characterization and the analysis of tumor cells within a microfluidic chip with hydrodynamic focusing capabilities.

Traditional histology is the gold standard for tissue visualization but is limited in providing 3D information. Combining 3D virtual histology (3DVH) with immunofluorescence (IF) to visualize structure and molecular expression is promising but not yet well established. In ocular pathologies, structures like the meibomian glands (MGs) have intricate 3D organizations that change in diseases. Understanding neurogenic impacts requires visualizing MG innervation using molecular markers, which 2D histology cannot achieve. Using our LIMPID optical clearing, we developed 3DVH for multiplexed large fields-of-view imaging of the ocular system, including MGs and the cornea. We overcame challenges like high melanin levels and the difficulty of clearing meibum. By combining 3DVH with IF, we visualize neural connections to MGs altered during dry eye disease. This tool is ideal for visualizing complex 3D tissue structures and molecular expressions, particularly effective for studying MGs and their innervation, which undergo significant changes previously extrapolated from 2D images.

Optical diffraction tomography (ODT) provides 3D refractive index (RI) information of biological specimens, revealing their structural, mechanical, and biochemical characteristics. To extend the limits of imaging through highly scattering samples, we integrate multi-wavelength illumination into the traditional ODT framework, which typically relies on illumination angle diversity. We explore illumination strategies that combine tilted angles with diverse wavelengths to enhance the accuracy and quality of 3D refractive index reconstructions.

Digital Holography is an imaging method based on interferometry that reconstructs in 3 dimensions both the amplitude and the phase of an electromagnetic field. Since second harmonic generation is a coherent process, it can studied with holography and offers a unique insight in complex, 3D samples with single-shot, label-free and specific microscopy of samples with nonzero second order susceptibilities. With an added multiplexed-polarization study, we collect structural information in biological tissues such as organization of collagen fibrils in porcine cornea.

Osteoarthritis (OA) causes structural and functional alterations in articular cartilage, which can be studied using label-free non-linear optical microscopy. However, this technique is limited by its small field of view (FOV). This becomes apparent when imaging curved surfaces such as cartilage covering joint condyles. To increase the effective imaging area, we propose a solution that involves measuring a three-dimensional surface profile of the sample using a deep learning-based method and aligning the sample surface to the scanning plane using a software-controlled goniometer platform. With this, we can increase the overall FOV and improve the efficiency of imaging studies of OA models.

Image Scanning Microscopy (ISM) super-resolution microscopy has gained momentum for its almost instantaneously improved resolution capabilities. Multiphoton ISM utilizes the shorter emission wavelength and confocal acquisition by exploiting each pixel on the camera as a pinhole, along with numerical enhancement, to achieve sub-diffraction-limit resolution. We present the use of a multiplexed approach for signal acquisition using a regular EMCCD camera. A spatiotemporal modulation scheme is employed to direct the ultrafast laser pulses to select foci within a field-of-view. Combined with a novel image reconstruction method, we show that only 49 images are required to achieve a resolution of 100 nm.

3D microscopy has become increasingly popular in the last few decades for applications in biology and medicine. For example, multi-photon microscopy is widely adopted for in vivo neural imaging for its cellular resolution, minimal invasion, optical sectioning and deep imaging performance. Super-resolution microscopy allows observation and studying of subcellular structures with a spatial resolution beyond diffraction limit of conventional fluorescence microscopes. In many applications, remote focusing (RF) is desirable to avoid sample perturbation especially in neuro imaging where minimal mechanical disturbance can be tolerated. Taking advantage of Adaptive Optics (AO), an aberration compensation technique that controls wavefront of excitation and/or fluorescence emission, we can remotely change imaging position at the same time of compensating for optical aberrations at depth. We will present the implementations of AO and RF in various microscopes that provide a high-speed disturbance-free approach for 3D deep imaging for various applications. We will show the limitation and the extension of the function and capability of 3D microscopy techniques for biological applications.

To process mesoscopic microscopy data, a new approach is needed. This is due to the size of the datasets, ranging from 15 to 25GB. Traditional methods used in neuroinformatics often fail on these datasets due to memory constraints or prohibitively long computational times. We present a toolkit that alleviates these problems and provides high-level access to commonly used functions to analyse whole mouse brain acquisitions. We have adopted the Open Microscopy Environment’s (OME) Next Generation File Format (NGFF) to store the data. This breaks the data into chunks, allowing for easier processing with lower computation times and less memory usage. Furthermore, the toolkit provides functions for image registration, image parcellation, image segmentation using deep learning, and brain template creation. We aim to apply the toolkit in group studies to investigate changes in cerebrovascular growth in mice exposed to hypoxia in the perinatal period.

This study evaluates the integration of a Tunable Acoustic Gradient-Index Lens (TAGLens) into laser scanning confocal microscopy (LSCM) for fast axial scanning and extended depth of field. A white light source was collimated and transmitted through the TAGLens, with reflected light focused onto a photomultiplier tube (PMT) via a pinhole aperture. Driven at 70 kHz, the TAGLens induced light intensity fluctuations analyzed by Fourier analysis, showed a distinct peak at the driving frequency. Results demonstrated that TAGLens modulates the conjugate plane, suggesting necessary compensations to maintain confocality and enable efficient fast volumetric imaging with extended depth of field in LSCM.

Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Here, we develop a scattering simulator that models low-contrast target signals obscured by background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length, without domain adaptation methods. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model’s generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.

Image Scanning Microscopy (ISM) enables good signal-to-noise ratio (SNR), super-resolution and high information content imaging by leveraging array detection in a laser-scanning architecture. However, the SNR is still limited by the size of the detector, which is conventionally small to avoid collecting out-of-focus light. Nonetheless, the ISM dataset inherently contains the axial information of the fluorescence emitters. We leverage this knowledge to achieve computational optical sectioning without sacrificing the conventional benefits of ISM. We invert the physical model to fuse the raw dataset into a single image with improved sampling, SNR, lateral resolution, and optical sectioning. We validate our approach with experimental images of biological samples acquired with a custom setup equipped with a single photon avalanche diode (SPAD) array detector. Furthermore, we generalize our method to other imaging techniques, such as multi-photon excitation fluorescence microscopy and fluoresce lifetime imaging. To enable this latter, we take advantage of the single-photon timing ability of SPAD arrays, accessing additional sample information.

The optical coherence tomography microscope (OCM) is expected to provide a few millimeters of imaging depth. However, the high numerical aperture for cellular-level resolution results in only a-few-micrometers depth-of-focus (DOF). Although computational refocusing can sharpen the image, the confocality still limits the imaging depth. Additionally, the inevitable speckle significantly reduces the effective resolution. To overcome these limitations, we demonstrate a no-pinhole spatially coherent full-field OCM hardware with several holographic signal processing methods. The hardware provides the in-focus lateral resolution of 1.7 µm but its DOF is only 5.5 µm. However, the absence of the confocal pinhole and additional phase-only spatial frequency filtering enable post-acquisition extension of the imaging depth up to a few millimeters. Furthermore, newly introduced speckle-reduction method significantly improved the effective lateral resolution. The high-contrast and cellular-level volumetric imaging of cancer spheroids is demonstrated over the full depth of the sample (around 0.3 mm).

Confocal reflectance microscopy shows a periodic bright and dark pattern when imaging collagen fibrils, however this phenomenon has not been explained. In addition, collagen fibrils below the diffraction limit are difficult to measure using traditional optical microscopy. This works aims to investigate a new confocal microscopy method which uses a known boundary as a reference to determine a collagen fibrils diameter. FDTD analysis has shown that a collagen fibrils diameter may directly affect the collected confocal signal, even below the Abbe diffraction limit. In addition, the use of multiple wavelengths may allow for distinction of collagen fibrils that may appear similar with one excitation, but differ in reflected intensity for another. We believe this new confocal interference method may give insight not only in measuring collagen fibril diameters, but may assist in explaining the periodic pattern that appears on collagen fibrils on many microscopy modalities.

Fluorescence Lifetime Imaging Microscopy (FLIM) systems are frequently prohibitively expensive, requiring expensive detectors, precise timing electronics, and advanced imaging systems. The substantial cost greatly restricts access to FLIM technology, confining its capabilities to well-funded research institutions and larger industries. In collaboration with STMicroelectronics, we present a cost-effective alternative utilising the STMicroelectronics VL53L8 sensor, a miniature commercial Time-of-Flight (ToF) sensor costing approximately $1, that can be reconfigured to achieve FLIM. Preliminary results suggest that this approach delivers results comparable to those of traditional commercial FLIM systems. Future research could integrate this detector technology into 3D printed microscopes for ultimate portability and cost reduction.

Fluorescent Confocal Microscopy (FCM) has revolutionized biology and medicine by enabling high-resolution, three-dimensional imaging of cells and tissues, advancing our understanding of disease mechanisms and facilitating the development of targeted therapies. However, effective utilization of FCM in many fields is limited by photobleaching/phototoxicity and its relatively slow imaging speed due to point-by-point scanning. Here we report on development of a scanless confocal microscopy technique with light sheet excitation. The core innovation behind this work is utilization of the broadband nature of fluorescence expression to eliminate the need for scanning light across the sample which, in return, significantly lowers the instrumentation complexity and the cost of the system. With such an approach, scanless performance is achieved by taking advantage of a narrow slit (optical encoding in one sample direction) and spectral encoding of the broadband fluorescent emission (spectral encoding in other sample direction). Results from Zemax numerical modeling, experimental characterization tests, and confocal imaging of HeLa and HEK293 cells will be presented.

We designed, built, and tested a microscope using the Linnik interferometer. The measurement of wide-field interference patterns employs a single-shot method, using light polarization manipulation and detection with a polarized camera. A low-coherence light-emitting diode serves as the light source, offering a resolution of 10 nm in the Z direction and diffraction-limited resolution in the X and Y directions. This single-shot method is robust against vibration and enables observation of moving objects. The instrument's simplicity and affordability render it highly valuable for diverse applications.

There is commercial interest in producing high quality microscopy images for low cost. Our research combines Fourier optics and machine learning to attempt to correct aberrations in low-quality fluorescence microscopy images. We use Fourier optics to generate a synthetic dataset and validate the approach and then show how the technique might be applied to real world fluorescence images.

The Plug-and-Play (PnP) methods enhance imaging by integrating advanced pre-trained denoisers within optimization schemes, improving reconstruction speed and quality. We investigate applying PnP-ADMM, a method using the Alternating Direction Method of Multipliers, to a novel tunable SIM (TSIM) system. The TSIM system, with a quasi-monochromatic source and a Wollaston prism, allows independent control over lateral and axial resolutions using only 9 raw images, a 40% data reduction compared to traditional 3D-SIM. Our approach, incorporating Total Variation (TV) regularization and a DnCNN denoiser, effectively preserves lateral resolution and suppresses noise. Results show the PnP-based method achieves theoretical TSIM resolution in both noiseless and 17dB noise simulations.

This study explores Alzheimer's disease (AD)-related optical signatures in the human retina using optical coherence tomography (OCT). Two OCT retinal datasets were generated: one from participants diagnosed with AD and another from participants without a diagnosis, but either with or without a family history of AD. A weakly supervised Transformer-based framework using multiple instance learning was developed to classify OCT images. Preliminary results showed promising classification accuracy for diagnosed AD cases but lower performance for family history-based classification. The findings demonstrated the potential of OCT as a non-invasive, accessible tool for detecting optical retinal signatures associated with AD.

Currently existing imaging theories of classical and cutting-edge microscopes do not share a universal framework. Here we describe a universal image formation theory covering almost all types of classical and cutting-edge microscopes such as bright-field, fluorescence, coherent Raman microscopes, and optical coherence tomography, i.e., linear, nonlinear, coherent, and incoherent microscopes. This theory is characterized by the following features: The formulation is four-dimensional and treats the space and time universally. The time-dependent interaction between the sample and the light is described by generalizing the quantum Liouville equation. The type of light-matter interactions and their impact on the imaging properties are described by double-sided Feynman diagrams. The coherent and incoherent interactions are universally formulated by exploiting the vacuumed field. The rigid imaging formulation of an individual imaging method can be derived by an established mathematical protocol. It enables comprehension of classical and cutting-edge microscopes in a single theoretical framework.