This PDF file contains the front matter associated with SPIE Proceedings Volume 11703, including the Title Page, Copyright information and Table of Contents

Introduction to SPIE Photonics West OPTO conference 11702: AI and Optical Data Sciences II.

Coherent Ising Machines (CIMs) are an emerging class of computational architectures that embed hard combinatorial optimization problems in the continuous dynamics of a physical system with analog degrees of freedom. While crisp theoretical results on the ultimate performance and scaling of such architectures are lacking, large-scale experimental prototypes have begun to exhibit promising results in practice. Our team at Stanford has begun to study the fundamental properties of CIM dynamics using a combination of techniques from statistical physics, random matrices, and dynamical systems theory. Many connections to recent work in neuroscience and deep learning are noted. Our work focuses specifically on CIMs that utilize the nonlinear threshold behavior of optical parametric oscillators to effect a soft (potentially glassy) transition between linear and binary dynamical regimes.

Photonics-based AI processors have the potential to outperform digital AI accelerators in terms of throughput, latency, and efficiency, and meet the growing demands of AI edge computing. One such class of processors, known as photonic reservoir computers, are a promising candidate for performing real-time edge computing, but their performance has been limited by the small number of effective nodes in the reservoir. We discuss our advances in developing a next-generation photonic reservoir with dual time and wavelength multiplexing. This allows for a reservoir with increased size and complexity, enabling it to successfully perform complex classification and prediction tasks.

Many real-time cognitive sensing signal processing and control applications require low SWAP edge processors with ultra-low latency adaptation and learning capabilities along with strict throughput, accuracy and power requirements. Achieving 3rd generation AI capabilities, i.e., real-time contextual adaptation, requires fast adaptive inference operations at low power beyond what is achievable with currently available neural networks and deep learning systems. While there has been tremendous progress in the form of edge accelerators, today’s processors lack capabilities for real-time processing, adaptation to novel situations, and low latency decision making. This paper addresses currently unsolved critical challenges in real-time cognitive sensing and autonomous system control applications, such as ultra-wide bandwidth and real-time signal denoising, anomaly detection, blind signal separation, and adaptive system equalization and control. We also present experimental results for low Cost – Size Weight and Power (C-SWAP) hardware implementation of an edge processor prototype implemented on a commercially available FPGA board.

Implementation of some signal processing algorithms on hardware has generally an advantage of efficiently implementation of complex processing. However, it still has some difficulties of developing natural optical phenomena because of various trade-off relation. Since these difficulties do not always allow a photonic hardware to emulate such an intermediary processing, further little assistances are necessary to complete the gap bridge and various machinelearning would play a significant role there. We discuss machine-learning-aided photonic hardware implementation incorporating natural optical phenomena with an example of a spectroscopic inspection technique for low cost, high speed, large data, and high spectral resolution.

Augmented reality devices, as smart glasses, enable users to see both the real world and virtual images simultaneously. Recently, waveguides incorporating holographic optical elements have been utilized in various grating structures to reduce the size and weight of the smart glasses, instead of bulky free-space headsets. However, these devices typically still possess physical and performance limitations, which lead to limited display quality and high cost. Here we report metasurface optical elements as a platform solution to these shortcomings. Through careful control of dispersion in the excited propagation and diffraction modes, we design a high-resolution waveguide display to the human eye box, with large field-of-view and in a full-color implementation. Supported by our combination of analytical and numerical simulations, we examine a metasurface optical waveguide with high input-output efficiency and high-resolution, implemented in a single metasurface layer.

Combinatorial problems, such as the Ising problem, are hard to solve with conventional electronics. Photonic systems have recently been proposed as an efficient platform to solve these problems faster and more efficiently, thus calling for the development of featured algorithms to run on photonic machines. A few recent findings, including the Photonic Recurrent Ising Sampler, a photonic machine that recurrently solves arbitrary Ising problems, will be presented in this talk, along with their experimental realizations in various platforms.

Here we introduce a novel amplitude-only Fourier-optical processor paradigm capable of processing large-scale ~(1,000 × 1,000) matrices in a single time-step and 100 microsecond-short latency. We exemplary realize a convolutional neural network (CNN) performing classification tasks on 2-Megapixel large matrices at 10 kHz rates, which latency-outperforms current GPU and phase-based display technology by one and two orders of magnitude, respectively. Training this optical convolutional layer on image classification tasks and utilizing it in a hybrid optical-electronic CNN, shows classification accuracy of 98% (MNIST) and 54% (CIFAR-10).

Spin Glasses (SG) are paradigmatic models for physical, computer science, biological and social systems. The problem of studying the dynamics for SG models is NP-hard, i.e., no algorithm solves it in polynomial time. Here we implement the optical simulation of an SG, exploiting the N segments of a Digital Micromirror Device to play the role of the spin variables, combining the interference at downstream of a scattering material to implement the random couplings and measuring the transmitted light intensity to retrieve the system energy. We demonstrate that This optical platform beats digital computation for large-scale simulation (N<12000).

Coherent Ising machines have been proposed as a promising platform for combinatorial optimization. Initial fiber-based, FPGA-assisted instantiations experimentally outperform quantum annealers based on superconducting qubits in speed and energy-efficiency due their ability to have programmable all-to-all connectivity between the Ising nodes. Since then, multiple flavors of coherent Ising machines have been proposed based on silicon photonics. In this talk, we will compare and contrast these integrated Ising machines with their table-top setup counterparts and their upcoming competitors in digital and analog electronics. Moreover, we will explain how large-scale problems can be mapped to small-scale integrated Ising cores.

In the post-Moore’s law era, dedicated digital accelerators have played an indispensable role in the success of artificial neural networks (ANNs) and their applications in artificial intelligence (AI). As the complexity of ANNS grows, analog electrical and optical computing are being considered as alternatives for achieving sustainable scalability and energy efficiency. In this talk we will present photonic accelerators that can potentially provide orders of magnitude increases in scalability and energy efficiency towards brain-like AI with hundred billion-neuron neural processing power.

We introduce a diffractive optical neural network architecture that can all-optically implement various functions, following the deep learning-based design of passive layers that work collectively. We created 3D-printed diffractive networks that implement all-optical classification of images of handwritten digits and fashion products as well as the function of an imaging lens, spectral filters, wavelength demultiplexers and ultra-short pulse shapers at terahertz part of the spectrum. This passive diffractive network framework is broadly applicable to different parts of the electromagnetic spectrum, including the visible wavelengths, and can perform at the speed of light various complex functions that computer-based neural networks can implement, and will find applications in all-optical image analysis, feature detection and object classification, also enabling new camera designs and optical components that perform unique tasks using diffractive networks designed by deep learning.

Spatial Ising Machines are simple optical acceleration devices enabling hard combinatorial optimization with millions of spins. We review our results, including noise acceleration and adiabatic evolution.

Analog neural networks are promising candidates for overcoming the sever energy challenges of digital Neural Network processors. However, noise is an inherent part of analogue circuitry independent if electronic, optical or electro-optical integration is the target. I will discuss fundamental aspects of noise in analogue circuits and will then introduce our analytical framwork describing noise propagation in fully trained deep neural networks comprising nonlinear neurons. Most importantly, we found that noise accumulation can be very efficiently supressed under realistic hardware conditions. As such, neural networks implemented in analog hardware should be very robust to internal noise, which is of fundamental importance for future hardware realizations.

As the ultra-high frequency nature of lightwave corresponds to great potential in wideband signal processing, nextgeneration electronic information systems of surveillance, radar and communications is promised with photonic signal processing systems. However, the sophisticated photonic systems suffer from various hardware defects, which severely limit the performance of signal processing. By introducing the emerging deep learning technology into the photonic system, the hardware defects can be recovered by the trained neural networks. Using different modified neural networks, we have demonstrated high-accuracy photonic analog-to-digital converters, Brillouin instantaneous frequency measurement, and high-fidelity photonic radar receivers. The demonstrated systems with simple configurations can outperform the conventional photonic processing system with complex configurations. Note that the adoption of neural networks may cause additional time delay to the signal flow. Photonic neural network accelerators (PNNs) become a promising solution to realize real-time signal processing. We propose and experimentally demonstrate several system architectures of photonic convolutional neural networks. The photonic dot product unit architecture implements the basic operation in convolution neural networks. And an optical patching scheme is demonstrated to enhances the power efficiency of the input ports in PNNs. Performance evaluations show that the proposed PNN architectures possess potential advantages of energy efficiency and computational power. We believe that, by combining the technical advantages of photonic signal processing and PNN acceleration, intelligent photonic signal processing systems with high-performance real-time wideband signal processing capabilities can be realized. Moreover, the large-scale photonic integration technology promises the fabrication of such hybrid systems in the future.

A photonic reservoir computer (RC) leverages optical phenomena to implement multiplication by large pseudo-random matrices used by reservoir computers to perform complex machine learning tasks. Here we show that the equations for propagation around a multimode (MM) ring resonator can be cast exactly in the standard RC form with speckle mixing performing the matrix multiplication, an optical nonlinearity, and optical feedback. The hyperparameters are the outcoupling efficiency, the nonlinearity saturation level, and the input bias. The MM ring geometry reduces the sampling rate of backend ADCs by the number of neurons compared to single mode rings and removes the costly optical-to-electrical conversions required at each time step in the arrays. Simulations show a ring using a strongly guiding 50-m planar waveguide gives 200 neurons and excellent predictions and classifications of Mackey-Glass waveforms, while a weakly guiding MM 200-m diameter fiber gives about 4,000 neurons and excellent predictions of chaotic solutions of the Kuramoto-Sivashinsky equation. We perform several simulations of both systems to demonstrate the spatial sampling requirements for the output speckle patterns and that these ring resonator RCs are not excessively sensitive to tuning of the hyperparameters. Finally, we propose designs implementing the system as a chip-scale device or with discrete components and a MM optical fiber.

We report on photonic crystal lasers (PCSELs) with high power and high beam quality. The PCSELs have been developed with AI-assisted technologies. The developed devices with a 500µm diameter successfully oscillated with a high, 10W-class peak output power and a very narrow divergence angle of 0.1°. This fact indicates that the devices operate in a complete single lateral and longitudinal mode even over the large area of 500µm diameter. The devices have been installed in time-of-flight LiDAR system. Very high-resolution operation has been successfully realized even though a lens system is not utilized, clearly demonstrating the advantage of high-brightness PCSELs.

Using a convolutional neural network to develop an optimal sampling strategy for LIDAR remote sensing. Detecting the distance to object is important for autonomous vehicles, surveying, and other remote sensing applications. LIDAR detects distances using a pulsed laser and a time-of-flight system to measure the position of all objects in a scene, however they are limited in the maximum distance they can measure due to low signal return. A convolutional neural network has been used to develop a sampling basis to effectively sample the scene, and also the reconstruction algorithm to recreate the 3D scene.

Artificial intelligence is attracting attention as a promising tool to create new clinical value from big data acquired by medical instruments. Here, I introduce results of employing deep learning while analyzing data of biological samples. In the classification of cell images acquired by imaging flow cytometer, deep learning enabled the rapid discrimination of a few cancer cells with an accuracy of 99% or more, in a milieu of thousands of normal blood cells. In addition, the application of AI to vibrational spectroscopy for the identification of amino acids or peptides with high sensitivity and accuracy, is also introduced.

We demonstrate highly accurate and fast cell imaging and classification systems enabled by the combination of disordered optical fiber for data transport and deep convolutional neural networks (DCNNs) for data analysis. Disordered optical fiber feature unique light transport properties based on the principle of transverse Anderson localization. A dense network of single mode-like transmission channels results in high spatial resolution while providing robustness regarding bending and environmental changes. DCNNs optimized for cell image reconstruction or cell classification have been trained and applied to perform rigorous testing. We show artifact-free real-time image reconstruction and >90% correct classification of cell samples.

Self driving cars are expected to drive safely under all weather conditions and traffic scenarios. Significant progress has been made so far, but further advancements in hardware and software are required to achieve this goal. This talk will discuss how lidar performance impacts autonomous driving; and how multiple sensing modalities combined with perception software are resolving edge cases experienced on the road.

Characterizing crystallographic orientation is essential for assessing structure-property relationships in crystalline solids. While diffraction methods have dominated this field, low throughput and high cost limit their applicability to small, specialized samples and restrict access to well-funded research institutions. We present a complementary optical technique that expands applicability and broadens access. This technique—which we call directional reflectance microscopy (DRM)—relies on acquiring a series of optical micrographs of chemically etched crystalline materials under different illumination angles. Using machine learning to correlate the directional reflectivity of the surface with the local etch-induced surface structure, DRM enables previously impossible crystal orientation mapping of large-scale, complex parts—such as entire multi-crystalline silicon solar cells, turbine blades, and complex parts produced by additive manufacturing technology. The simplicity, low cost, and enhanced sample throughput of our method promise to expand the availability of crystallographic orientation mapping significantly, making it readily available in education as well as academic research and industrial settings.

Optical Wireless Power Transmission (OWPT) is one of the technologies of wireless energy transfer. OWPT uses optical light source as an energy emitter, and solar panel as an energy detector. With current trends of availability of large laser power source at lower prices and sophisticated image recognition technologies, OWPT increases its attention. In order to obtain high energy transfer efficiency in OWPT, precise control of laser illuminating position is required. We apply co-axially aligned optical system of imaging function and laser beam steering function to satisfy the requirement. At the presentation, we will introduce our approach of OWPT.

This is the presentation video for (Short) journey beyond freeform

We present three different approaches to apply deep learning to inverse design for nanophotonic devices. The forward models use device parameters as inputs and device responses as outputs. This model works as a fast approximation method which can be integrated in the optimization loop, and can accelerate the optimization. The network is updated as we obtain more simulation data on the fly for better approximation. The inverse modeling uses a network trained with the device responses as inputs, and the device parameters as outputs. This way the network outputs the device structure given the target optical response. This network can also be updated as we obtain more data during the optimization and validation. The generative model we use is a variant of a conditional variational autoencoder, and the network learns the statistical characteristics of the device structure, and it generates a series of improved designs given the target device responses. By using these three models, we demonstrate how to design nanophotonic power splitters with multiple splitting ratios.

Holographic displays are considered to be promising technologies for augmented and virtual reality devices. Using spatial light modulators (SLMs), they can directly modulate the wavefront of light. Through the modulation of the wavefront, they can provide observers three-dimensional imagery. However, they suffer from a large computation load, and it is important to overcome the disadvantage for the popularization of holographic display techniques. In this invited paper, we adopt a deep learning algorithm for the fast generation of computer-generated holograms (CGHs). We propose the deep neural network designed for the generation of complex holograms. The overall algorithm for the learning-based generation of CGHs using the network is introduced, and the training strategy is provided. The simulation and experimental results are demonstrated, and we verified the feasibility of using the deep learning algorithm for CGH computation.

Blazed and slanted gratings present particularly interesting optical properties in that they can diffract light predominantly into one side of the diffraction plane (m≤0 or m≥0). This is useful in a variety of applications, especially for AR display combiners, where slanted gratings are commonly used to improve efficiency to the eyebox. Nanoimprint lithography (NIL) has been explored as a route to mass manufacture AR waveguides with slanted structures. However, NIL presents several challenges associated with the process of molding and releasing angled features in a high refractive-index, functional material. In this paper we report a series of breakthroughs achieved at Facebook Reality Labs (FRL) that enable replication of gratings with a slant angle up to 60° and an aspect ratio of 10:1 in a material with refractive index higher than 1.90. To the best of our knowledge, these results are the first public demonstration of highly-slanted gratings imprinted in such a RI material.

We analyze the information processing capacity of coherent optical networks formed by trainable diffractive surfaces to prove that the dimensionality of the solution space describing the set of all-optical transformations established by a diffractive network increases linearly with the number of diffractive surfaces, up to a limit determined by the size of the input/output fields-of-view. Deeper diffractive networks formed by larger numbers of trainable diffractive surfaces span a broader subspace of the complex-valued transformations between larger input/output fields-of-view, and present major advantages in terms of their function approximation power, inference accuracy and learning/generalization capabilities compared to a single diffractive surface.

We report a broadband diffractive optical network that can simultaneously process a continuum of wavelengths. To demonstrate its success, we designed and experimentally validated a series of broadband networks to create single/dual passband spectral filters and a spatially-controlled wavelength de-multiplexer that are composed of deep learning-designed diffractive layers to spatially and spectrally engineer the output light. The resulting designs were 3D-printed and tested using a terahertz time-domain-spectroscopy system to demonstrate the match between their numerical and experimental output. Broadband diffractive networks diverge from intuitive/analytical designs, creating unique optical systems to perform deterministic tasks and statistical inference for machine learning applications.

Almost all modern day imaging systems rely on digital capture of information. To this end, hardware and consumer technologies strive for high resolution quantization based acquisition. Antithetical to folk wisdom, we show that sampling quantization noise results in unconventional advantages in computational sensing and imaging. In particular, this leads to a novel, single-shot, high-dynamic-range imaging approach. Application areas include consumer and scientific imaging, computed tomography, sensor array imaging and time-resolved 3D imaging. In each case, we present a mathematically guaranteed recovery algorithm and also demonstrate a first hardware prototype for basic digital acquisition of quantization noise.

The generation of an optical supercontinuum with short (fs) input pulse duration is a highly complex process that exhibits rich nonlinear dynamics. Here, we show that one can teach a machine learning model to learn the nonlinear dynamics of ultrashort pulse propagation and predict the full-field propagation dynamics of supercontinuum based only on the input pulse characteristics (peak power, duration and chirp).

Through evolution, nature has developed the most energy efficient method for achieving a certain goal. In contrast to the traditional hand-crafted algorithms pervasive in computational imaging, our approach takes inspiration from nature, and in particular optical physics, to craft qualitatively new algorithms. Here, we show that propagation of light through an optical medium has properties that can be exploited for development of a new class of image and video enhancement algorithms that offer unprecedented improvements in performance over conventional counterparts. In certain cases, these algorithms also have the potential to be implemented physically compared to synthetically fashioned computational techniques in practice today.

Although the successes of artificial intelligence in areas such as automatic translation are well known, the application of the powerful techniques of deep learning to current optics research is at a comparatively early stage. However, an area with particular promise for deep learning to accelerate both basic science and applications is in ultrafast optics, where nonlinear light-matter interactions lead to highly complex dynamics, including the emergence of extreme events. In the particular field of nonlinear fibre optics, we have recently reported a number of results that have shown how deep learning can both augment existing experimental techniques as well as provide new theoretical insights into the underlying physics. The objective of this paper is to review a selection of our work in this area.

We demonstrate the use of machine learning for adaptive control of nonlinear optical systems. From deep learning algorithms to nonlinear control methods, the optical sciences are an ideal platform for integrating data-driven control and machine learning for robust, self-tuning operation. For the specific case of mode-locked lasers, commercially available servo-controllers enact a training and execution software module capable of self tuning the laser cavity even in the presence of mechanical and/or environmental perturbations and discrepancies, thus providing algorithmic stabilization of mode-locking performance. The execution stage quickly stabilizes optimal mode-locking using various algorithmic innovations including (i) extremum seeking control, (ii) deep reinforcement learning and (iii) deep model predictive control. The demonstrated methods are robust and equation-free, thus requiring no detailed or quantitatively accurate model of the physics.

We introduce our studies on optical nonlinearity compensation schemes based on machine learning technology for optical communication systems. We examine the performance of nonlinear equalization using an artificial neural network (ANN), by comparing our scheme with other schemes, including the Volterra series transfer function (VSTF) and the support vector machine (SVM). We describe an SVM-based nonlinear equalizer and our proposed improved version. We discuss the advantage of an ANN in a polarization tracking application. We also discuss the problem of computational complexity of a conventional VSTF-based nonlinear equalizer. We consider ways of improving the computational complexity of the VSTF. An ANN-based nonlinear equalizer shows advantages over the VSTF in terms of computational complexity.

With the advent of the 5G wireless networks, achieving tens of gigabits per second throughputs and low latency has become a reality. This level of performance will fuel numerous realtime applications where the computationally heavy tasks can be performed in the cloud. The increase in the bandwidth along with the use of dense constellations places a significant burden on the speed and accuracy of analog-to-digital converters (ADC). A popular approach to create wideband ADCs is utilizing multiple channels each operating at a lower speed in the time-interleaved pattern. However, an interleaved ADC comes with its own set of challenges. The parallel architecture is very sensitive to the inter-channel mismatch, timing jitter, clock skew between different ADC channels as well as the nonlinearity within individual channels. In this project, we utilize a deep learning algorithm to learn the complete and complex ADC behavior and to compensate for it.

Wavelength Division Multiplexing (WDM) is the key technology in ultra-high capacity links that form the backbone of the internet. Hundreds or more data channels each at a different wavelength travel through a single fiber resulting in aggregate data rates exceeding many Terabits per second. The fundamental limit to the data transmission rate is the optical crosstalk between channels induced by the inevitable nonlinearity of the fiber. Traditional methods for compensating for the reduction in the bit error rate caused by the crosstalk include numerical backpropagation as well as nonlinear Volterra filter, both implemented in the digital domain at the receiver. Backpropagation through the canonical nonlinear Schrodinger equation is computationally expensive and beyond the capability of today’s DSP at the data rates that optical networks operate. Volterra filters scale superlinearly with an increasing number of taps and which in turn scale with the amount dispersion in the fiber. Therefore, they are not the ideal solution for high data rates. In this talk, we report on the application of machine learning, and neural networks in particular, on the compensation of optical crosstalk in WDM communication. We compare the performance of different machine learning models such as support vector machine (SVM), decision tree, convolutional neural network (CNN) in terms of the achievable bit error rate on both binary and multilevel modulated data. We further evaluate the sensitivity of the error rate to the resolution of the analog to digital converter (ADC) and to the signal to noise ratio as well as the latency of our algorithms

Ultraviolet-visible (UV-Vis) spectroscopy is a well-established technique for real-time analyzing contaminants in finished drinking water and wastewater. However, it has struggled in surface water because surface water such as river water has more complex chemical compositions than drinking water and lower concentrations of nutrient contaminants such as nitrate. Previous spectrophotometric analysis using absorbance peak at UV region to estimate nitrate in drinking water performs poorly in surface water because of interference from suspended particles and dissolved organic carbon which absorb light along similar wavelengths. To overcome these challenges, the paper develops a machine learning approach to utilize the entire spectral wavelengths for accurate estimation of low concentration of dissolved nutrients from surface water background. The spectral training data used in this research are obtained by analyzing water samples collected from the US-Canada bi-nationally regulated Detroit River during agricultural seasons using A.U.G. Signals' dual channel spectrophotometer system. Confirmatory concentrations of dissolved nitrate in these samples are validated by laboratory analysis. Several commonly used supervised learning techniques including linear regression, support vector machine (SVM), and deep learning using convolutional neural network (CNN) and long short-term memory (LSTM) network are studied and compared in this work. The results conclude that the SVM with linear kernel, CNN with linear activation function, and LSTM network are the best regression models, which are able to achieve a cross validation root-mean-squared-error (RMSE) less than 0.17 ppm. The results demonstrate effectiveness of the machine learning approach and feasibility of real-time UV-Vis spectral analysis to monitor dissolved nutrient levels in the surface watersheds.

With the rapid development of integrated nanophotonics technology, the circuit architecture for optical neural networks that construct neural networks based on integrated nanophotonics has been widely investigated. Since the optical neural network can complete the inference processing just by propagating the optical signal in the network, the inference processing by the optical neural network is expected more than one order of magnitude faster than the electronics-only implementation of an artificial neural network (ANN). In this paper, we first propose an optical vector-matrix multiplication (VMM) circuit using wavelength division multiplexing, which enables inference processing with ultra-wideband. Although the optical VMM circuit is extremely fast, the initial version is designed for fully connected network structures, which consume large amounts of power in laser light sources. As a solution to this power explosion, this paper especially proposes sparsely connected network structures for the optical VMM operation, which reduces the power consumed in the laser sources by three orders of magnitude without any speed and bandwidth degradation compared with the fully connected counterpart. Although the main part of ultra-fast ANN is VMM, batch normalization and activation function are integral parts for accurate inference processing in ANN. Batch normalization applied at inference processing is a technique for improving the inference accuracy of ANN. Without batch normalization and activation function, the inference accuracy of ANN may significantly degrade. In this paper, we next propose electronic circuit implementation of batch normalization and activation function, which significantly improves the accuracy of inference processing without sacrificing the speed performance of inference processing. Those two functions can be implemented based on an energy-efficient O-E-O repeater circuit. We specifically propose the electronic implementation of Exponential Linear Unit (ELU in short) as an activation function. It is known that ELU largely contributes to improving the inference accuracy of ANN as well as learning speed. Finally, in this paper, we demonstrate the ultra-fast and accurate operation of the optical-electronic ANN circuit using TensorFlow and optoelectronic circuit simulator.

This report on research in progress demonstrates a machine learning (ML) approach to array-based free-space optical communication using mobile devices. Spatial codes are transmitted using arrays of lasers or light emitting diodes for increased resilience and throughput, and ML models are trained on the channel alphabet to provide efficient decoding at the receiver. Various ML models, transmission array configurations, and spatial codes are compared for performance, and a proof-of-concept system is demonstrated. ML decoding of spatial symbols under noisy/perturbed channel conditions was successfully accomplished, however significant challenges are identified with throughput on mobile devices. Future experimentation is outlined to incorporate testing over greater distances under more realistic conditions.

We present a diffractive network, trained for pulse engineering to shape input pulses into desired optical waveforms. The synthesis of square-pulses with various widths was experimentally demonstrated with 3D-fabricated passive diffractive layers that control both the amplitude and phase profile of the input terahertz pulse across a wide range of frequencies. Pulse-width tunability was also demonstrated by altering the layer-to-layer distances of a diffractive network. Furthermore, the modularity of this framework was demonstrated by replacing part of an already-trained network with newly-trained layers to tune the width of the output terahertz pulse, presenting a Lego-like physical transfer learning approach.

We report a deep-learning based compact spectrometer. Using a spectral encoder chip composed of unique plasmonic tiles (containing periodic nanohole-arrays), diffraction patterns created by the transmitted light through these tiles are captured by a CMOS sensor-array, without the use of any lenses or other components between the plasmonic encoder and the CMOS-chip. A neural network rapidly reconstructs the input light spectrum from the recorded lensless image data, which was blindly tested on randomly-generated new spectra to demonstrate the success of this computational on-chip spectrometer, which will find applications in various fields that demand low-cost and compact spectrum analyzers.

Using deep learning-based training of diffractive layers we designed single-pixel machine vision systems to all-optically classify images by maximizing the output power of the wavelength corresponding to the correct data-class. We experimentally validated our diffractive designs using a plasmonic nanoantenna-based time-domain spectroscopy setup and 3D-printed diffractive layers to successfully classify the images of handwritten-digits using a single-pixel and snap-shot illumination. Furthermore, we trained a shallow electronic neural network as a decoder to reconstruct the images of the input objects, solely from the power detected at ten distinct wavelengths, also demonstrating the success of this platform as a task-specific, single-pixel imager.

In this paper, we introduce a concept of the hybrid lens as a novel form of optical information processing apparatus that integrates the conventional optical lenses and the recently proposed neural lens, that is, an image post-processing technique based on generative convolution neural networks (GCNN). This integration is based on leveraging the fact that the Fourier plane is the common working principle of both types of lens. We demonstrate how manipulating the coherent light's spectral components on the Fourier plane behind a biconvex lens is a computationally-free alternative to performing convolution matrix operation in GCNN, which involves high computational expenses. In our approach, the GCNN can perform image generation in a shorter time. Our hybrid lens only requires computational power at the levels that the embedded resources on medical devices can afford. This feature is very important for commercialization as it allows making standalone units like microscopes without relying on external resources such as computing clouds.

We succeeded in reducing image acquisition time while maintaining image quality by applying a profile sensor (PS) to compressed imaging (CI). Single pixel imaging (SPI) has been proposed as a method to reconstruct a two-dimensional image using only a point detector. However, SPI’s disadvantage is a long acquisition time. We applied a PS to CI and show that it reduces acquisition time. A PS is an area sensor with pixels arranged in a two-dimensional array, but the output is a projection of the image in the x and y directions.

Reservoir computing has emerged as a lightweight, high-speed machine learning paradigm. We introduce a new optoelectronic reservoir computer for image recognition, in which input data is first pre-processed offline using two convolutional neural network layers with randomly initialized weights, generating a series of random feature maps. These random feature maps are then multiplied by a random mask matrix to generate input nodes, which are then passed to the reservoir computer. Using the MNIST dataset in simulation, we achieve performance in line with state-of-the-art convolutional neural networks (1% error), while potentially offering order-of-magnitude improvement in training speeds.

Flat optics allow the production of integrated, lightweight, portable and wearable optical devices. In this work we propose a flat optics design platform that employs concepts from evolutionary algorithms to deep learning with convolutional neural networks, and demonstrate a general design framework that can furnish an arbitrarily designed system response in as little as 50nm of silicon. The proposed framework is fundamental for our most recent experimental paper, in which we present a plethora of high efficiency devices, including, but not limited to: polarizing beam splitters, dichroic mirrors and metasurfaces for a novel 2-pixel display technology.

We report an approach assisted by deep learning to design spectrally-sensitive multi-band absorbers that work in the visible range. We propose a five-layered metal-insulator-metal grating structure composed of aluminum and silicon dioxide, and design its structural parameters by using an artificial neural network (ANN). For a spectrally-sensitive design, spectral information of resonant wavelengths is additionally provided as input as well as the reflection spectrum. The ANN facilitates highly robust design of grating structure that has an average mean squared error of 0.023. The optical properties of the designed structures are validated using electromagnetic simulations and experiments. Analysis of design results for gradually-changing target wavelengths of input show that the trained ANN can learn physical knowledge from data. We also propose a method to reduce the size of the ANN by exploiting observations of the trained ANN for practical applications.

In recent decades, with the rapid development of image sensor technology, image acquisition has gradually evolved from a single sensor mode to a multi-sensor mode. The data information obtained by a single sensor is limited, and the use of multi-source data fusion can provide a more accurate understanding of the observation scene. This paper proposes a network structure of infrared visible color night vision image fusion based on deep learning. The network adopts a fusion-encoding-decoding structure for end-to-end learning to achieve the purpose of color night vision image fusion, making the image more in line with human visual effects. The fusion structure contains a multi-scale feature extraction block and a channel attention block, which perform feature extraction on low-resolution infrared images and visible images respectively. The multi-scale feature block can expand the receptive field and avoid losing too much feature information. The channel attention block can improve the sensitivity of the network to channel characteristics. A certain number of convolutional layers and deconvolutional layers are used in the network to realize the encoding and decoding of the feature map to achieve the purpose of restoring the color fusion image. After experimental verification, our method has a good fusion effect, rich colors, and conforms to the human visual effect.