Share Email Print

Proceedings Paper

Deep learning for analysis and synthesis of dense and multicolor localization microscopy (Conference Presentation)
Author(s): Elias Nehme; Eran Hershko; Lucien E. Weiss; Tomer Michaeli; Yoav Shechtman

Paper Abstract

Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we address two fundamental problems of localization microscopy using machine learning: emitter density, and color determination. Modern microscopy can produce images of biological specimen at very high (super) resolution, by precisely determining the positions of numerous blinking light emitting molecules over time. To achieve fast acquisition time, a high density of molecules is required, which poses a significant challenge in terms of image processing. Existing approaches use elaborate algorithms with many parameters that require tuning and a long computation time. Here, we report an ultra-fast, precise, and parameter-free method for super-resolution microscopy that utilizes deep-learning: by feeding the computer images of dense molecules along with their correct positions, it is trained to automatically produce super-resolution images from blinking data. Next, we demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing single-molecule methods for spectral classification require additional optical elements in the emission path, e.g. spectral filters, prisms, or phase masks, our neural net correctly identifies static as well as mobile emitters with high efficiency using a standard, unmodified single-channel configuration – based on inherent chromatic aberrations in a standard microscope. Finally, we demonstrate how deep learning can be used to design phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species. While point-spread-function engineering for spectral differentiation has been implemented in various applications in recent years, the optimal way to design such a PSF remains unclear. Here, we use a neural net to perform such design automatically, directly optimizing the desired cost function, namely, simultaneous localization and color detection of point emitters.

Paper Details

Date Published: 27 July 2019
Proc. SPIE 11060, Optical Methods for Inspection, Characterization, and Imaging of Biomaterials IV, 110600U (27 July 2019); doi: 10.1117/12.2525372
Show Author Affiliations
Elias Nehme, Technion-Israel Institute of Technology (Israel)
Eran Hershko, Technion-Israel Institute of Technology (Israel)
Lucien E. Weiss, Technion-Israel Institute of Technology (Israel)
Tomer Michaeli, Technion-Israel Institute of Technology (Israel)
Yoav Shechtman, Technion-Israel Institute of Technology (Israel)

Published in SPIE Proceedings Vol. 11060:
Optical Methods for Inspection, Characterization, and Imaging of Biomaterials IV
Pietro Ferraro; Simonetta Grilli; Monika Ritsch-Marte; Christoph K. Hitzenberger, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?