Emerging Topics in Artificial Intelligence (ETAI) 2024

As the rapid increase in the amount of information, there has been increasing demand on developing domain-specific computing hardware capable of addressing complex tasks that general-purpose von-Neumann computers cannot readily address. Here I show a kind of unconventional computing schemes, a probabilistic computing, which is promising to address various computationally hard problems, in particular the computational tasks for which probabilistic algorithms are often applied. Stochastic magnetic tunnel junction, an unconventional spintronics device utilizing the thermally activated random magnetization switching, plays a key role. I first describe the concept of the probabilistic computer with the stochastic magnetic tunnel junction and then show several proof-of-concepts of the probabilistic computer. I also discuss device physics and engineering for performing large-scale problems within a limited computation time.

3D Optical Neural Networks (ONN) is a promising solution to the energy, time, and area yearning AI hardware. The 3D additive manufacturing technique with Two-Photon Polymerization can be used to build these 3D dense ONNs. The hybrid waveguide circuit which fuses the polymer and air-clad waveguides is the important interconnect for the ONN. The polymer-cladded waveguide can support single mode and evanescent coupling while the air-clad can support tight bend for dense integration.

We report an all-optical phase conjugation (OPC) approach utilizing diffractive wavefront processors collectively optimized via deep learning. Our approach uses purely passive materials for OPC, thus obviating the need for external power sources or digital processing. Moreover, the resulting design is notably compact, axially spanning ~84 wavelengths between the input and output planes. We experimentally demonstrated its proof-of-concept using the terahertz spectrum, proving its efficacy in performing phase conjugation of waves perturbed by unknown distortions. Our method is expected to provide transformative OPC solutions that can be integrated into diverse imaging and sensing systems at different parts of the spectrum.

Reservoir computing is a promising framework for signal processing applications, but the optimization of such physical reservoirs remains an important challenge in the field. In this work we address this challenge with a new technique based on the use of a delayed input and we test it using an experimental optoelectronic reservoir. We demonstrate that this technique can replace the standard hyperparameters optimisation of reservoirs with a much simpler approach based on the scan of only two parameters. We test this approach on tasks of different nature and in various operating condition of the optoelectronic reservoir, confirming its superiority to the standard hyperparameters scan.

Artificial intelligence based on ever-increasing computing power including neuromorphic computing has heralded a disruptive horizon in many ways of our life. The concept of the diffractive neural network harvesting the spatial connection among digitised pixels is one of the promising emerging technologies in this race as it offers a potential for ultrafast neuromorphic computing power with a high neural density. The key to this high performance is the connectivity provided by the optical links among the diffractive layers. Therefore, we have witnessed the successful operation of the diffractive neural works operating beyond the spatial domains including spectral and polarisation multiplexing. In this talk, I will show diffractive neural networks multiplexed in the optical orbital angular momentum and as well as temporal domains.

Nanophotonic structures are crucial for controlling light at scales smaller than its wavelength. While designing for linear polarization is straightforward, creating nanostructures for helically structured light, like circularly polarized light and optical vortices, is challenging due to complex near-field chiral interactions with matters in helical electromagnetic fields. In this presentation, we apply topology optimization, an intelligent design approach, to create 3D nanogap antenna structures with outstanding chiroptical functionalities. With these structures, we demonstrate giant chiral dissymmetry (up to g = 1.70), polarization conversion around the Poincaré sphere, and circularly polarized far-field emission from a linear dipole embedded within the gap. Additionally, our in-depth analysis reveals a physical connection between the flow of spin angular momentum of light within the nanostructure and the local density of optical chirality. The insight, combined with our developed structures, offers a fresh perspective for designing chiral nanophotonic structures for better control of chiral molecules using light.

Photonics, a multidisciplinary field that encompasses the study and manipulation of light, has witnessed profound transformations thanks to machine learning (ML) techniques. ML models, as deep neural networks, can be used to accelerate the design flow in nanophotonics. However, a large number of data samples can be needed to train and validate deep neural networks and therefore the potential speed-up in a design flow offered by these models can be drastically reduced. Some methods that can be used to achieve high modeling accuracy without requiring a high amount of data samples will be presented. For example, the concepts of response features, feasible design space, and cross-fertlization of supervised and unsupervised learning will be discussed. ML models can be then more compact, more interpretable, and built with a reduced amount of data samples. In addition, a transfer learning approach to accelerate the generation of inverse models will be discussed. Also, some metasurface-based examples based on diverse nanofabrication technologies, such as two-photon polymerization and e-beam lithography, will be illustrated.

I will overview our recent work testing the absolute limits of optical computing, including realizations of optical neural networks that use vastly less than one photon per multiplication, operating deep in the stochastic regime dominated by quantum noise. I will also discuss how optical neural networks scale up, and how they may offer advantages (including 3-5 orders of magnitude more energy-efficient inference) for implementing large language models. Works referenced: Ma, S. Y., Wang, T., Laydevant, J., Wright, L. G., & McMahon, P. L. (2023). Quantum-noise-limited optical neural networks operating at a few quanta per activation. arXiv preprint arXiv:2307.15712 Anderson, M. G., Ma, S. Y., Wang, T., Wright, L. G., & McMahon, P. L. (2023). Optical transformers. arXiv preprint arXiv:2302.10360. Wang, T., Ma, S. Y., Wright, L. G., Onodera, T., Richard, B. C., & McMahon, P. L. (2022). An optical neural network using less than 1 photon per multiplication. Nature Communications, 13(1), 123.

In recent years, micro- and nanofluidic channels for single nanoparticle detection have attracted much attention. we fabricate a nanofluidic channel with a width of about 500 nm on a quartz-glass surface that crosses a pair of parallel microfluidic channels printed on a PDMS surface. Nanoparticles dispersed in an electrolyte solution on the microchannel are inducted along the microchannel and into the nanochannel by flow control of the other microchannel. Transport of these nanoparticles is recorded using a high-speed CMOS camera and the trajectories are analyzed by sing particle tracking analysis. As a result, the target nanoparticles are effectively pulled into the opening of nanofluidic channel with appropriate transport velocity overcoming thermal fluctuations. Furthermore, the nanoparticle detection frequency is improved by the flow rate difference between the microchannels. The present method is expected to contribute to the optical manipulation technology of single nanoparticles in liquids in micro- and nanochannels.

Integrated photonics offers the potential to realize complex nonlinear photonic systems. I will describe recent work on the synchronization of coupled Kerr microresonators that can be exploited for beam combining, spectral reshaping, and optical frequency division.

This talk will consider the future of metal halide perovskite (MHP) photovoltaic (PV) technologies as photovoltaic deployment reaches the terawatt scale. The requirements for significantly increasing PV deployment beyond current rates and what the implications are for technologies attempting to meet this challenge will be addressed. In particular how issues of CO2 impacts and sustainability inform near and longer-term research development and deployment goals for MHP enabled PV will be discussed. To facilitate this, an overview of current state of the art results for MHP based single junction, and multi-junctions in all-perovskite or hybrid configurations with other PV technologies will be presented. This will also include examination of performance of MHP-PVs along both efficiency and reliability axes for not only cells but also modules placed in context of the success of technologies that are currently widely deployed.

The recent advent of robust, refractory (having a high melting point and chemical stability at temperatures above 2000°C) photonic materials such as plasmonic ceramics, specifically, transition metal nitrides (TMNs), MXenes and transparent conducting oxides (TCOs) is currently driving the development of durable, compact, chip-compatible devices for sustainable energy, harsh-environment sensing, defense and intelligence, information technology, aerospace, chemical and oil & gas industries. These materials offer high-temperature and chemical stability, great tailorability of their optical properties, strong plasmonic behavior, optical nonlinearities, and high photothermal conversion efficiencies. This lecture will discuss advanced machine-learning-assisted photonic designs, materials optimization, and fabrication approaches for the development of efficient thermophotovoltaic (TPV) systems, lightsail spacecrafts, and high-T sensors utilizing TMN metasurfaces. We also explore the potential of TMNs (titanium nitride, zirconium nitride) and TCOs for switchable photonics, high-harmonic-based XUV generation, refractory metasurfaces for energy conversion, high-power applications, photodynamic therapy and photochemistry/photocatalysis. The development of environmentally-friendly, large-scale fabrication techniques will be discussed, and the emphasis will be put on novel machine-learning-driven design frameworks that leverage the emerging quantum solvers for meta-device optimization and bridge the areas of materials engineering, photonic design, and quantum technologies.

Artificial intelligence (AI) techniques have boosted the capability of optical imaging, sensing, and communication. Concurrently, photonics facilitate the tangible realization of deep neural networks, offering potential benefits in terms of latency, throughput, and energy efficiency. In this talk, I will discuss our efforts in AI photonics with two examples. The first involves employing a convolutional neural network for achieving single-shot full-field measurement of optical signals. The second example pertains to implementing a deep neural network with a multiple-scattering system featuring structural nonlinearity, thereby enabling nonlinear computations using linear optics.

With the proliferation of networked sensors and artificial intelligence, there is an increasing need for edge computing where data is processed at the sensor level to reduce bandwidth and latency while still preserving energy efficiency. In this talk, I will discuss how meta-optics can be used to implement computation for optical edge sensors, serving to off-load computationally expensive convolutional operations from the digital platform, reducing both latency and power consumption. I will discuss how meta-optics can augment, or replace, conventional imaging optics in achieving parallel optical processing across multiple independent channels for identifying, and classifying, both spatial and spectral features of objects.

Minimally invasive endoscopy using multicore fibers shows great potential for numerous applications in biomedical imaging. With a diffuser on the distal side of the fiber and image recovery using AI, single-shot 3D imaging is possible by encoding the image volume into 2D speckle patterns. In comparison to equivalent lens systems, a higher space-bandwidth product can be achieved. However, decoding the image with iterative algorithms is time-consuming. Thus, we propose utilizing a neural network for fast 2D and 3D image reconstruction at video rate. In this work, single-shot 3D fluorescence imaging with physics-informed neural network is presented, which is promising for calcium imaging at in vivo brain diagnostics.

We present a fast virtual-staining framework for defocused autofluorescence images of unlabeled tissue, matching the performance of standard virtual-staining models using in-focus label-free images. For this, we introduced a virtual-autofocusing network to digitally refocus the defocused images. Subsequently, these refocused images were transformed into virtually-stained H&E images using a successive neural network. Using coarsely-focused autofluorescence images, with 4-fold fewer focus points and 2-fold lower focusing precision, we achieved equivalent virtual-staining performance to standard H&E virtual-staining networks that utilize finely-focused images, helping us decrease the total image acquisition time by ~32% and the autofocusing time by ~89% for each whole-slide image.

Optical manipulation and imaging play critical roles in biomedical applications, however, applying these technologies to hard-to-reach regions remains challenging. We introduce a series of innovative AI-driven methods designed to facilitate both high-fidelity light field control and image reconstruction through a lensless multicore fiber with an ultra-thin diameter below 0.4 mm. Our approach enables precise, controlled rotation of human cancer cells around all three axes, enabling 3D tomographic reconstructions of these cells with isotropic resolution. Moreover, we developed deep neural networks tailored for quantitative phase imaging through the lensless fiber endomicroscope by efficiently decoding phase information from speckles captured on the fiber's distal end. The integration of these advanced optical and computational techniques culminates in a powerful optical fiber probe, capable of sophisticated optical manipulation and phase imaging, offering new perspectives for optical manipulation and endoscopic imaging.

Micro (less than five millimeters in length) and nano (less than one micron) plastics exist in marine and terrestrial habitats and are detrimental to both the environment and animal health. A comprehensive study is necessary to understand their origin and distribution and to find environmental and human protection strategies. Recently, optical and Raman tweezers have proved to be an efficient tool to trap, manipulate and characterize micro and nano plastics. In this work, trapping configurations of micro and nano plastic particles have been investigated computationally. By combining the rigorous calculation based on the Transition matrix method with approaches based on machine learning, we investigate a broad range of dimensions and optical properties enhancing the efficiency of computations and unlocking new avenues for advancement in microplastics research.

Optical tweezers manipulate microscopic objects with light by exchanging momentum and angular momentum between particle and light, generating optical forces and torques. Understanding and predicting them is essential for designing and interpreting experiments. Here, we focus on geometrical optics and optical forces and torques in this regime, and we employ neural networks to calculate them. Using an optically trapped spherical particle as a benchmark, we show that neural networks are faster and more accurate than the calculation with geometrical optics. We demonstrate the effectiveness of our approach in studying the dynamics of systems that are computationally “hard” for traditional computation.

Brain cells are complex and delicate materials with unique physical properties that can be influenced by physical stress. Recent studies have shown that physical stress affects the biological response of molecules inside living cells. We have been developing a single-objective light-sheet microscope and optical manipulation tools to investigate the dynamic properties of living brain cells. This integration allows us to observe 3D biological processes with high precision while enabling the manipulation and characterization of individual cells and microcompartments. In this presentation, I will explain the operating principles of multimodal optical force microscopy, share preliminary results, and discuss future applications.

We analyse the collective behaviours of Escherichia coli (E. coli) active matter. The individual movements of these E. coli can be accurately tracked and examined using a recently developed machine learning software called DeepTrack (Midvedt et al., 2021). This provides greater insight into the chaotic dynamics of E. coli swarms as well as the potential to critically assess current theoretical models. DeepTrack analysis can also be applied in more complex environments including interactions with microstructures made with photolithography. Analysing the movements of E. coli active matter with DeepTrack has promising implications in engineering and biomedical applications.

We examine motility-induced phase separation (MIPS) in two-dimensional run and tumble disk systems using both machine learning and noise fluctuation analysis. Our measures suggest that within the MIPS state there are several distinct regimes as a function of density and run time, so that systems with MIPS transitions exhibit an active fluid, an active crystal, and a critical regime. The different regimes can be detected by combining an order parameter extracted from principal component analysis with a cluster stability measurement. The principal component-derived order parameter is maximized in the critical regime, remains low in the active fluid, and has an intermediate value in the active crystal regime. The different regimes can also be characterized via changes in the noise power of the fluctuations in the average speed. Similar methods can be applied to active matter in the presence of substrates produced via optical means or for situations in which the particles are being manipulated optically.

The perhaps most widely used tool for measuring forces and manipulating particles at the micro and nano-scale are optical tweezers which have given them widespread adoption in physics, chemistry and biology. Despite advancements in computer interaction driven by large-scale generative AI models, experimental sciences—and optical tweezers in particular—remain predominantly manual and knowledge-intensive, owing to the specificity of methods and instruments. Here, we demonstrate how integrating the components of optical tweezers—laser, motor, microfluidics, and camera—into a single software simplifies otherwise challenging experiments by enabling automation through the integration of real-time analysis with deep learning. We highlight this through a DNA pulling experiment, showcasing automated single molecule force spectroscopy and intelligent bond detection, and an investigation into core-shell particle behavior under varying pH and salinity, where deep learning compensates for experimental drift. We conclude that automating experimental procedures increases reliability and throughput, while also opening up the possibility for new types of experiments.

Light-powered microrobotic swarms, excelling in manipulation efficiency and motion pattern customization, are pivotal for micro-fabrication and biomedical applications. Herein, we introduce an integrated platform capable of autonomously transporting light-powered microrobotic swarms over long distances within complex environments. The embedded real-time feedback control algorithm ensures swarm integrity and facilitates the adept navigation around unpredictable obstacles. The successful operation of both trapping- and nudging-based microrobots prove the versatile applicability of our platform.

Information processing is vital for living systems and involves complex networks of active processes. These systems have influenced various forms of modern machine learning, including reservoir computing. Reservoir computing utilizes networks of nodes with fading memory to perform computations and make complex predictions. Reservoirs can be implemented on computer hardware or unconventional physical substrates like mechanical oscillators, spins, or bacteria, known as physical reservoir computing. We demonstrate physical reservoir computing with a synthetic active microparticle system that self-organizes from an active and passive component into inherently noisy nonlinear dynamical units. The self-organization and dynamical response of the unit is the result of a delayed propulsion of the microswimmer to a passive target. A reservoir of such units with a self-coupling via the delayed response can perform predictive tasks despite the strong noise resulting from the Brownian motion of the microswimmers. To achieve efficient noise suppression, we introduce an architecture that uses historical reservoir states for output. We discuss the node and collective reservoir dynamics.

We developed a computational model to simulate contours of entangled lambda DNA, considering various imaging conditions and DNA labeling densities. These simulations were used to generate super-resolution DNA images for training a deep neural network (ANNA-PALM) to reconstruct DNA contours from localization images. Our approach enabled reliable contour prediction from microscopy images under diverse imaging noise levels and dye labeling densities. Analysis of experimental data revealed bright and dark DNA segments, potentially linked to local microviscosity effects on dye photoswitching kinetics. Our integrated computational modeling and deep learning workflow enables precise mapping of entanglement loci and detailed quantification of their nanoscale dynamics, providing key insights into how topological constraints shape polymer motion in diverse materials.

The report exhibits autonomous navigation of a microparticle in noisy environment aided by real time detection and machine learning enabled feedback mechanism. Specifically, we use Actor-Critic Reinforcement Learning (ACRL) algorithm for navigation of a microparticle to a specific target location under the inevitable influence of Brownian motion as well as flow fields. Furthermore, we investigate a multi-agent reinforcement learning model (MARL) for navigation of multi particle systems where specific target locations and interparticle separation are provided as sensory inputs. Our study demonstrates that, the model trained on a noisy environment can learn effective policy for navigation, reaching various target states after limited number of exploration episodes.

Whole Slide Images (WSIs) provide rich information for the diagnosis and treatment planning of various cancers. However, the giant size of WSIs brings quite challenges in image classification task. The conventional deep learning based classification framework involves partitioning WSIs into patches and using patch feature extractors and classifiers to make image classification predictions. However, these methods face challenges: 1) Processing a large number of patches incurs high computational and memory costs, and 2) A substantial number of patches contain redundant information. To overcome these challenges, we develop a novel method to automatically select representative patches from which informative can be gleaned for image classification by leveraging a large pre-trained deep learning model to select a subset of patches to represent the WSIs. Subsequently, we build an end-to-end deep learning model to jointly learn informative features from patches and integrate all patches with attention for classifying the WSIs. The experimental results have demonstrated that our approach improves the classification performance compared with state-of-the-art deep learning methods.

Graphs are used to model complex relationships in various domains, such as interacting particles or neural connections within a brain. Efficient analysis and classification of graphs pose significant challenges due to their inherent structural complexity and variability. Here, a novel approach is presented to address these challenges through the development of a graph autoencoder. The proposed autoencoder effectively summarizes graph structures while preserving important topological details through multiple hierarchical pooling steps. This enables the extraction of physical parameters describing the graphs. The performance of the network is demonstrated across diverse datasets originating from complex systems. This approach holds great promise for examining diverse systems, enhancing our comprehension of various forms of graph data.

In the current artificial intelligence (AI) framework, the explainability of AI is buried under the black-box nature of the implicit implementation of the latent feature space of an AI architecture. However, the explainability of AI can be enhanced by explicitly defining its feature space, quantifying the similarity (or dissimilarity) and orthogonality (or non-orthogonality) properties between its feature vectors, and extracting common features (projection onto subspaces) of its feature vectors. This paper presents an approach that defines a theoretically infinite family of features space (tIFFS) that uniquely combines the distinctive properties of inner product and orthogonality operations between feature vectors, and the projection of feature vectors onto subspaces in a Hilbert space that is formed by infinite dimension function space (IDFS). Simulations show there exists a random forest classifier that can be trained on tIFFS with the high classification performance accuracy. Simulations also show the tIFFS can capture suitable inner product, orthogonality, and projection properties in the IDFS Hilbert space.

We introduce a quantitative measure of inference uncertainty broadly applicable to existing neural networks on inverse imaging problems. We first established forward-backward cycles between the input and target image domains, and derived uncertainty estimators from cycle consistency between successive cycles. We theoretically showed the regression relationship between cycle consistency and the derived uncertainty estimators and validated their effectiveness by fitting a simple linear regression model for quantitative uncertainty prediction. We applied our method to image deblurring and super-resolution neural networks to successfully recognize unseen distribution shifts during the blind testing stage, outperforming other baseline methods.

This paper proposes a unique concept of colony of artificial intelligence (AI). It also defines a terminology, “marriage of AI-agents” to allow marriage between two AI-agents in the colony of AI to produce unique offspring. When two AI-agents marry they exchange model weights (called AI-genetic information) through the crossover technique of genetic algorithm (while randomizing the biases to become flexible on the decisions) to produce a child AI-agent. Mutation technique is used to make minor changes to the AI-genetic information of the child AI-agent. Simulations are conducted towards building a colony of AI using a pretrained VGG16 model and the CIFAR10 dataset. Current simulations show that the flexibility of a parent (not of both parents) improves the performance of the child AI-agent by 6% (from 72% to 78%) on average with 10 epochs.

Fluorescence microscopy is the gold standard for investigation of biological samples. While powerful, this technology has its limitations, including phototoxicity, technical difficulties in introducing fluorescent markers, and limited simultaneous labeling of different structures due to the need for spectral separation. Here, artificial intelligence-driven virtual staining can provide label-free imaging. Deep neural networks are trained to learn the correlation between a label-free image and a ground-truth fluorescence image. However, the main challenge for the transition from traditional to virtual staining is the acquisition of huge datasets including labeled and unlabeled data. We have performed proof-of-concept experiments to investigate the possibilities of transfer learning in virtual staining. Therefore, U-Net architectures were pretrained on a larger dataset and later on trained again on a smaller and more specific dataset. We show that transfer learning decreases the needed dataset size and may even improve prediction quality. Furthermore, the interpretability of the trained networks was studied.This was investigated using guided backpropagation and modified input images.

We introduce a deep learning-based approach utilizing pyramid sampling for the automated classification of HER2 status in immunohistochemically (IHC) stained breast cancer tissue images. Our deep learning-based method leverages pyramid sampling to analyze features across multiple scales from IHC-stained breast tissue images, managing the computational load effectively and addressing the challenges of HER2 expression heterogeneity by capturing detailed cellular features and broader tissue architecture. Upon application to 523 core images, the model achieved a classification accuracy of 85.47%, demonstrating the ability to counteract staining variability and tissue heterogeneity, which might improve the accuracy and timeliness of breast cancer treatment planning.

During radiotherapy, the target location undergoes changes attributed to physiological factors, such as respiration. Ensuring treatment accuracy necessitates a real-time, noninvasive image guidance model for continuous monitoring. Optical body surface guidance boasts advantages like radiation-free operation, a large irradiation field, and real-time monitoring of body surface alterations. However, it falls short in acquiring internal tumor information. In contrast, X-ray imaging provides internal details but is constrained by its radiation dosage and limitations in real-time acquisition. To address these challenges, we integrated the strengths of X-ray and optical body surface information. A patient-specific intra- and extra-vivo correlation model is employed to establish a correlation between the optical body surface and the internal tumor. Additionally, we employ a dual flat-panel X-ray intermittent supervision model. This combined approach enables real-time, non-invasive tracking of the target area and facilitates personalized radiotherapy.

This presentation covers the opportunities and challenges of photonic-electronic chip-based machine intelligence acceleration hardware. We will start with a review of device-level component performance specifications such as footprint, energy consumption, reconfiguration speed, for instance. Emerging materials, when integrated monolithically into photonic waveguide circuits, show promise for next generation opto-electronic components offering high FOMs, however have high barrier to entry in for foundry PDKs. Beyond devices, we will explore a variety of architectural choices, known as co-design optimization. Parallelization strategies, smart routing, optical hardware function implementation (e.g. Fourier transformation on-chip) will be covered. Next, we explore chip packaging options including ADK and digital-twin hardware in the loop optimizations thereof. Finally, examples of prototyping will be shared and application options discussed.

This study presents a novel method for learning representations from chest X-rays using a memory-driven transformer-based approach. The model is trained on a low-quality version of the MIMIC-CXR dataset, which contains 377,110 chest X-rays from 65,379 patients and 227,827 imaging studies. The model uses a relational memory to record crucial information during the generation process and a memory-driven conditional layer normalization technique to integrate this memory into the transformer's decoder. The model is divided into distinct sets for training, validation, and testing. We aim to establish an intuitively comprehensible quantitative metric, through vectorization of the radiology report. This metric leverages the learned representations from our model to classify 14 unique lung pathologies. The F1-score measures classification accuracy, indicating the model's viability in diagnosing lung diseases. The model's potential applications extend to more robust performance in radiology report generation.

This research explored using Mutual Connectivity Analysis with local models for classifying Autism Spectrum Disorder (ASD) within the ABIDE II dataset. The focus was on understanding brain region differences between individuals with ASD and healthy controls. We conducted a Multi-Voxel Pattern Analysis (MVPA), using a data-driven method to model non-linear dependencies between pairs of time series. This resulted in high-dimensional feature vectors representing the connectivity measures of the subjects, used for ASD classification. To reduce dimensionality, we used Kendall’s τ coefficient method, preparing the vectors for classification using a non-linear kernel-based SVM. We compared our approach with methods based on cross-correlation and Pearson correlation. The results are consistent with current literature, suggesting our method could be a useful tool in ASD research. Further studies are required to refine our method.

Deep learning has emerged as a powerful tool in medical imaging, fundamentally transforming the field by automatically learning intricate patterns and features from large datasets. Its application spans various tasks such as image classification, segmentation, registration, reconstruction, and synthesis across diverse modalities including MRI, CT, PET/CT, PET/MRI, ultrasound, and X-ray.

Functional magnetic resonance imaging data suggest that temporal correlations in cortical networks vary across distinct behavioral states, providing for the dynamic reorganization of patterned activity. However, these data generally lack the temporal resolution to establish links between cortical signals and the continuously varying fluctuations in spontaneous behavior observed in awake animals. To understand the relationship between neural activity, functional connectivity and behavior, we used wide-field mesoscopic calcium imaging to monitor cortical dynamics in awake mice. I will present an approach we developed to quantify rapidly time-varying functional connectivity. Our "graph of graphs" approach, relying on Riemannian geometry, extracts the latent dynamics of time-varying functional connectivity in neuronal networks. Our analysis demonstrates that spontaneous behaviors are represented by fast changes in the correlational structure of cortical networks that are distinct from fluctuations in the magnitude of network activity. Combining mesoscopic imaging with simultaneous cellular-resolution two-photon microscopy demonstrated that correlations among neighboring neurons and between local and large-scale networks also encode behavior.

We present a rapid, stain-free, and automated viral plaque assay utilizing deep learning and time-lapse holographic imaging, which can significantly reduce the time needed for plaque-forming unit (PFU) detection and entirely bypass the chemical staining and manual counting processes. Demonstrated with vesicular stomatitis virus (VSV), our system identified the first PFU events as early as 5 hours of incubation and detected >90% of PFUs with 100% specificity in <20 hours, saving >24 hours compared to the traditional viral plaque assays that take ≥48 hours. Furthermore, our method was proven to adapt seamlessly to new types of viruses by transfer learning.

Accurate segmentation of blood vessels provides valuable insights into the underlying vascular structure, facilitating disease detection and treatment planning. Small vessel disconnection and low contrast are the most important problems in decreasing the small vessel segmentation performance. In our paper, we develop a novel deep learning model with a vessel attention module to enhance the network's ability by learning vessel boundary information and enhancing the image contrast around the vessel boundaries. Moreover, we introduce a prototype-based post-processing method to reconnect broken vessel segments, ensuring a fully connected vessel segmentation result. Experimental results on a large CT dataset have demonstrated that our method can obtain substantially improved vessel segmentation performance, particularly for small vessels which cannot be accurately identified by alternative deep learning methods under comparison.

The integration of artificial intelligence (AI) in healthcare, particularly using advanced deep learning and generative methods, presents substantial challenges and opportunities. Implementation of AI in clinical settings, such as radiology, requires evaluation beyond algorithm development. Measuring the effectiveness of AI is complex: should success metrics be based on the number of AI uses in radiology, the variety of pathologies analyzed, throughput, user engagement, or clinical quality measures like diagnostic accuracy or study turnaround time? Economic factors like return on investment are also critical. The author, drawing on multi-year practical experience in large-scale AI deployment in radiology, proposes methods to evaluate and enhance the value of AI in medical imaging. These include AI-PROBE, which assesses radiologists' performance with and without AI, and can provide advances like revised radiology Quality Assurance (QA) programs, or improved multi-institutional public health tracking. The presentation will conclude with introducing a multi-speaker session, covering in-house AI system development, the use of AI in clinical research, and FDA regulatory aspects.

Artificial intelligence in radiology has advanced rapidly over the last decade. Radiology is particularly amenable to AI because of its focus on imaging. In this presentation, I will review some of the latest advances in radiology AI, including the use of transformers and large language models. I will provide examples of radiology AI for diverse clinical applications including cardiovascular, cancer, and liver disease.

Rapid advances in artificial intelligence (AI) and machine learning have enabled rapid development of decision support tools for application in broad health care areas. Many of the AI systems target medical imaging applications. The availability of public datasets additionally stimulated the advancement of the medical AI systems and the ability of research groups to contribute to the field. In-house AI system development is crucial for exploring new ideas, and new research directions and applications which strongly contribute to the generation of fundamental knowledge by a broad research community in broad clinical areas. Number of in-house computer-aided AI decision support applications will be presented. It is essential to properly train and rigorously validate a clinical decision support tool and verify its generalizability and reliability prior to translation for patient care in the clinic. Best practices for the development and performance assessment of computer-aided AI decision support systems will be discussed.

Over 75% of all FDA-cleared software as a medical device relate to use cases in radiology. Despite this large prevalence, the current status-quo of training artificial intelligence (AI) tools entails using unimodal (imaging-only) algorithms. Moreover, retraining such models for new tasks requires using training supervised AI algorithms from scratch, using manually curated labels from scratch, even if it may be for the same modality or anatomy. In radiology, generating such labels requires expensive clinical expert time, limiting the development of capable AI models across tasks. In this presentation, I will describe the development and use of multi-modal vision-language models (VLMs) for radiological applications. VLMs present numerous benefits such as zero-shot classification, label-efficient adaptation to varying tasks, and improved robustness. Such new capabilities provided by VLMs is poised to usher in a new era of models for solving current and future challenges in radiology.

As the number and range of new AI medical devices continues to grow, they pose novel challenges for regulators, including the FDA, to balance safety and effectiveness while allowing for industry to quickly innovate. This presentation will provide an overview of the current regulatory environment for AI-enabled medical devices in radiology, gastroenterology, dermatology, and pathology. It will also include a description of some of the important advances coming from our regulatory science research here at FDA focused on improving and streamlining the development, assessment, and long-term monitoring of medical AI. Our efforts include research addressing the limitations of small and fragmented patient datasets, identifying AI bias and their sources, reducing algorithm performance differences across subpopulations, assessing novel and continuous-learning AI, developing assessment pipelines that account for factors such as heterogeneity in the reference standard, and developing effective post-market monitoring of AI devices.

Hyperdimensional computing is an emerging computing paradigm that leverages distributed representations of input data. The hyperdimensional distributed representations facilitate energy-efficient, low-latency, and noise-robust computations using low-precision and basic arithmetic operations. In this presentation, we introduce optical hyperdimensional distributed representations using laser speckles for adaptive, efficient, and low-latency optical in-sensor processing. We focus on applications for optical soft-touch interfaces and tactile sensors. We demonstrate that this optical approach achieves high accuracy in touch or tactile recognition while significantly reducing the amount of training data and computational burdens compared to traditional deep learning-based sensing approaches.

A high-performance photonic reservoir, which utilizes injection locking of a semiconductor multimode laser (SML), will be developed. This innovative design allows for fully parallel and high-bandwidth operation at telecommunication wavelength. The output of this system is projected in space and imaged onto a digital micromirror device, which provides a readout and facilitates the hardware integration of programmable output weights. By using a highly multimodal semiconductor laser, injection locking enables a large number of modes to be simultaneously locked to the high frequency modulated injection laser that provides the input signal, resulting in high dimensionality of the reservoir. The hardware integration of programmable output weights enables the system to be optimized for specific tasks, improving performance and reducing power consumption.

Reservoir computing (RC) is a neuromorphic machine learning paradigm that is ideal for temporal signal processing and is suitable for analog implementations in physical substrates, including photonic devices. RC has been extended to quantum systems due to the enhanced capabilities provided by an enlarged Hilbert space. In that regard, quantum reservoir computing (QRC) has the advantage of avoiding barren plateaus during training. Photonic architectures have already been studied for QRC applications. In our research, we propose a scalable quantum photonic platform for QRC that is suitable for solving temporal tasks. The physical substrate of our reservoir is an optical pulse, which recirculates through an optical cavity with losses, thus creating a quantum memory. The dissipation device (a beam-splitter) also allows the injection of external information and the weak monitoring of the reservoir. Our work focuses on the ability to process classical signals in real time by creating a physical ensemble of identical pulses inside a fiber and the noise robustness of our architecture by tuning the squeezing produced inside the optical cavity.

Disorder is a pervasive characteristic of natural systems, offering a wealth of non-repeating patterns. In this study, we present a novel storage method that harnesses naturally-occurring random structures to store an arbitrary pattern in a memory device. This method, the Stochastic Emergent Storage (SES), builds upon the concept of emergent archetypes, where a training set of imperfect examples (prototypes) is employed to instantiate an archetype in a Hopfield-like network through emergent processes. We demonstrate this non-Hebbian paradigm in the photonic domain by utilizing random transmission matrices, which govern light scattering in a white-paint turbid medium, as prototypes. Through the implementation of programmable hardware, we successfully realize and experimentally validate the capability to store an arbitrary archetype and perform recognition at the speed of light. Leveraging the vast number of modes excited by mesoscopic diffusion, our approach enables the simultaneous storage of thousands of memories without requiring any additional fabrication effort, moreover these memories can be grouped to realize higher level classes thus realizing patterns classification.

We present an all-optical image denoiser based on spatially-engineered diffractive layers. Following a one-time training process using a computer, this analog processor composed of fabricated passive layers achieves real-time image denoising by processing input images at the speed of light and synthesizing the denoised results within its output field-of-view, completely bypassing digital processing. Remarkably, these designs achieve high output diffraction efficiencies of up to 40%, while maintaining excellent denoising performance. The effectiveness of this diffractive image denoiser was experimentally validated at the terahertz spectrum, successfully removing salt-only noise from intensity images using a 3D-fabricated denoiser that axially spans <250 wavelengths.

Optical computing provides a compelling avenue to sustain the rapid growth of computing power needed by the rise of AI. However, it still struggles to efficiently implement the all-optical nonlinearities required to achieve effectively deep neural networks, a prerequisite for modern performance. Here, we exploit purely a linear optical setup to perform optical nonlinear mapping for information processing and compression. The essential ingredient is to encode information on a spatial light modulator embedded in a multiple scattering cavity. Light exiting the cavity encodes a much richer information than its linear counterpart. Fed into a very simple neural network as a digital decoder, we demonstrate its potential for various machine learning tasks include detecting pedestrians for self-driving cars, defining a new state-of-the-art in optical computing.

Coupled systems with multiple interacting degrees of freedom provide a fertile ground for emergent dynamics, which is otherwise inaccessible in their solitary counterparts. Particularly, nonlinearity and non-equilibrium dynamics enable new opportunities in coupled photonic systems that are not present in their linear and equilibrium counterparts that can have profound consequences in sensing and computing. In this talk, I will overview recent experimental progress on accessing such dynamics in time-multiplexed networks of nonlinear resonators towards computing and sensing applications. I will present demonstrations of topological dissipation, non-equilibrium spectral phase transitions, topological mode-locked lasers, non-Hermitian topologically enhanced sensing, and photonic elementary cellular automata. I will also overview the progress on integrated optical parametric oscillators (OPOs) and their networks in lithium niobate (LN) nanophotonics for classical and quantum information processing applications.

Fluorescence lifetime imaging microscopy (FLIM) measures fluorescence lifetimes of fluorescent probes to investigate molecular interactions. However, conventional FLIM systems often requires extensive scanning that is time-consuming. To address this challenge, we developed a novel computational imaging technique called light field tomographic FLIM (LIFT-FLIM). Our approach acquires volumetric fluorescence lifetime images in a highly data-efficient manner, significantly reducing the number of scanning steps. We demonstrated LIFT-FLIM using a single-photon avalanche diode array on various biological systems. Additionally, we expanded to spectral FLIM and demonstrated high-content multiplexed imaging of lung organoids. LIFT-FLIM can open new avenues in the biomedical research.

Raman spectroscopy (RS) is a real-time, label-free, and non-invasive spectral sensing technique to distinguish chemical compounds of underlying tissues or other substances. However, Raman scattering results in weak signals and must be manually processed before analysis can occur. This slows down experimentation and requires subjective human input at certain steps of processing, which can cause variation across datasets. Various machine-learning denoising methods have been proposed, but very few, if any, have been successful in providing an accurate end-to-end denoising algorithm that works on a broad dataset across various real-life use cases. In this paper, we propose an end-to-end spectral-based deep learning model for various photonics-related denoising tasks.

The traditional histochemical staining of autopsy tissue samples usually suffers from staining artifacts due to autolysis caused by delayed fixation of cadaver tissues. Here, we introduce an autopsy virtual staining technique to digitally convert autofluorescence images of unlabeled autopsy tissue sections into their hematoxylin and eosin (H&E) stained counterparts through a trained neural network. This technique was demonstrated to effectively mitigate autolysis-induced artifacts inherent in histochemical staining, such as weak nuclear contrast and color fading in the cytoplasmic-extracellular matrix. As a rapid, reagent-efficient, and high-quality histological staining approach, the presented technique holds great potential for widespread application in the future.

Machine learning tools, in particular artificial neural networks, have become ubiquitous in diverse applications. However, training such networks requires massive data sizes and produces a large carbon footprint. Inspired by neuromorphic computing ideas, we implement a novel programmable optical random neural network by positioning a scattering medium at the Fourier plane performing feature extraction on high-dimensional data. We genetically program the scattering medium to select the best random projection kernel among different orientations. Initial test accuracies are improved by 7-22% on various datasets using only 1% of the search space. Our simple and scalable method is a promising future computing alternative.

We present subwavelength imaging of amplitude- and phase-encoded objects based on a solid-immersion diffractive processor designed through deep learning. Subwavelength features from the objects are resolved by the collaboration between a jointly-optimized diffractive encoder and decoder pair. We experimentally demonstrated the subwavelength-imaging performance of solid immersion diffractive processors using terahertz radiation and achieved all-optical reconstruction of subwavelength phase features of objects (with linewidths of ~λ/3.4, where λ is the wavelength) by transforming them into magnified intensity images at the output field-of-view. Solid-immersion diffractive processors would provide cost-effective and compact solutions for applications in bioimaging, sensing, and material inspection, among others.

We introduce a diffractive super-resolution display system combining an electronic encoder and a diffractive decoder network to project super-resolved images using a low-resolution spatial light modulator (SLM). This deep learning-enabled display system achieves ~4x super-resolution, corresponding to a ~16x increase in the space-bandwidth product, which was also experimentally demonstrated using 3D-fabricated diffractive decoders that operate at the THz spectrum. The design principles of this diffractive super-resolution display were also used to project high-resolution color images using a low-resolution SLM. Diffractive super-resolution image projection paves the way for developing compact, low-power, and computationally efficient high-resolution image and video display systems.

Lucendi has developed Aqusens - an AI-based holographic imaging flow cytometer platform that includes a lens-less instrument enabling rapid screening of liquids and characterization of microobjects for many different applications. In contrast with conventional lens-based cytometers, Aqusens achieves much higher throughput and dynamic range, while built in a robust and portable package that can be used for in-lab or in-field deployments. Aqusens operations are enabled by different applications of neural networks that are used for autofocusing and phase recovery, as well as for advanced data analytics. The use of AI in Aqusens enables optimization of computing that allows practical deployments of the system for real applications.

Current approaches to optical neural networks are only focused on the arithmetic operations, such as matrix vector multiplication and in some cases the activation function. The control flow to enable large matrices, multiple connected layers or methods to increase precision are completely delegated to digital electronics. Here we present our approach to enabling all-optical control flow and precision control to enable full optical AI inference in the future, eliminating the need to jump between the electronic and optical domain for each layer, to increase performance and efficiency.

When an intense laser interacts with a gas of atoms, high-order harmonics are generated. In the time domain, this radiation forms a train of extremely short light pulses, of the order of 100 attoseconds. Attosecond pulses allow the study of the dynamics of electrons in atoms and molecules, using pump-probe techniques. This presentation will highlight some of the key steps of the field of attosecond science.

We present a method for accurately performing complex-valued linear transformations with a Diffractive Deep Neural Network (D2NN) under spatially incoherent illumination. By employing 'mosaicing' and 'demosaicing' techniques, complex data are encoded into optical intensity patterns for all-optical diffractive processing, and then decoded back into the complex domain at the output aperture. This framework not only enhances the capabilities of D2NNs for visual computing tasks but also opens up new avenues for applications in image encryption under natural light conditions to demonstrate the potential of diffractive optical networks in modern visual information processing needs.

We demonstrate a reconfigurable diffractive deep neural network (termed R‑D2NN) with a single physical model performing a large set of unique permutation operations between an input and output field-of-view by rotating different layers within the diffractive network. Our study numerically demonstrated the efficacy of R‑D2NN by accurately approximating 256 distinct permutation matrices using 4 rotatable diffractive layers. We experimentally validated the proof-of-concept of reconfigurable diffractive networks using terahertz radiation and 3D-printed diffractive layers, achieving high concordance with numerical simulations. The reconfigurable design of R‑D2NN provides scalability with high computing speed and efficient use of materials within a single fabricated model.

Optical neuromorphic computing marks a breakthrough over traditional digital computing by offering energy-efficient, fast, and parallel processing solutions while challenges remain in incorporating nonlinearity efficiently. Leveraging nonlinear wave dynamics in optical fibers as a computational resource may provide a solution. Our research demonstrates how femtosecond pulse propagation in optical fibers can emulate neural network inference, utilizing the high phase sensitivity of broadband light for creating nonlinear input-output mappings akin to Extreme Learning Machines (ELMs). Experimental results show high classification accuracies and low RMS errors in function regression, all at pico-joule pulse energy. This indicates our method's potential to lower energy consumption for inference tasks, complementing existing spatial-mode systems. We also investigated femtosecond pulses' nonlinear broadening effects – self-phase modulation and coherent soliton fission – demonstrating their distinct impacts on classification tasks and showcasing broadband frequency generation as a powerful, energy-efficient tool for next-generation computing.

We present GedankenNet, a self-supervised learning framework designed to eliminate reliance on experimental training data for holographic image reconstruction and phase retrieval. Analogous to thought (Gedanken) experiments in physics, the training of GedankenNet is guided by the consistency of physical laws governing holography without any experimental data or prior knowledge regarding the samples. When blindly tested on experimental data of various biological samples, GedankenNet performed very well and outperformed existing supervised models on external generalization. We further showed the robustness of GedankenNet to perturbations in the imaging hardware, including unknown changes in the imaging distance, pixel size and illumination wavelength.

Mode decomposition is a quantitative technique for analyzing multimode fibers. With pre-knowledge of the eigenmodes, the phase and amplitude weights of each mode can be extracted from the optical field. In this paper, we introduce a simple deep learning-based mode decomposition method by integrating a physical model with a deep neural network. We demonstrate that this method can decompose up to thousands of modes based on pure-intensity images.

Multiple scattering enables optical encryption: the input information (i.e., plaintext) is scrambled and only speckles (i.e., cyphertext) are output for access. The decryption, however, is ineffective with conventional methods. Recent intervention of deep learning (DL) decrypts information from speckles with much higher fidelity, driven by massive data. We developed a complex-valued platform to decrypt complex information (e.g., human face) from the speckles, with sufficient-high fidelity for face recognition. Continuous efforts endeavor for higher stability and security. By introducing a spin-multiplexing disordered metasurface as an ultra-stable speckle generator, the system demonstrates excellent decryption efficiency over extended periods in noisy environment with numerous encryption channels and a proposed double-secure scheme provides robust protection for the plaintext with a security key. Also, breaking inherent correlation among the speckles via a speckle modulation network can further boost the security and enable hierarchical authentication encryption. Collectively, speckle-based cryptosystem via DL is promising towards practice.

We introduce the enhanced Fourier Imager Network (eFIN), an end-to-end deep neural network that synergistically integrates physics-based propagation models with data-driven learning for highly generalizable hologram reconstruction. eFIN overcomes a key limitation of existing methods by performing seamless autofocusing across a large axial range without requiring a priori knowledge of sample-to-sensor distances. Moreover, eFIN incorporates a physics-informed sub-network that accurately infers unknown axial distances through an innovative loss function. eFIN can also achieve a three-fold pixel super-resolution, increasing the space-bandwidth product by nine-fold and enabling substantial acceleration of image acquisition and processing workflows with a negligible performance penalty.

We introduce an information hiding-decoding system, which employs a passive diffractive processor as the front-end and an electronic decoder as the back-end, offering a fast, energy-efficient, and scalable solution for protecting visual information. This diffractive processor all-optically transforms arbitrary input messages into deceptive output patterns, decipherable only through a jointly-trained electronic decoder neural network. This method can successfully hide infinitely many input messages into ordinary-looking patterns at its output field-of-view, which can be subsequently decoded by an electronic network. We experimentally validated the feasibility of our information-hiding camera by 3D-printing a physical diffractive system and testing it under terahertz illumination.

Photonic systems offer a promising platform for analog neuromorphic computing and machine learning acceleration, boasting advantages such as massive parallelism, low latency, and energy efficiency. Disordered photonic media have been utilized for implementing neural networks (NNs) architectures with simultaneous coding and processing of information, overcoming digital NNs' bottleneck of data transfer between memory and processor. I explore second-order nonlinear disordered photonic media assembled from oxide nanoparticles, particularly barium titanate and lithium niobate nanocrystals. Thanks to the simultaneous linear scattering and second-harmonic generation, these media enable multiple implementation of the activation function in the optical neural network, facilitating deep multi-layer operation. Experimental demonstrations showcase the potential of these media, particularly a deep two-layer optical neural network based on a nonlinear disordered multiple-scattering slab of lithium niobate nanocrystals, enhancing computing performance for various machine learning tasks including image classification and regression.

This talk will describe new microscopes and space-time algorithms that enable 3D or super-resolution fluorescence microscopy and phase measurement with high resolution on dynamic samples. Traditional model-based image reconstruction algorithms work together with neural networks to optimize the inverse problem solver and the data capture strategy in order to account for sample motion during the capture time of a multi-shot computational imaging method.

In this work we propose a new training scheme for analog neural networks. Instead of complexifying the training algorithm by trying to incorporate more information about the non-idealities of the system, we shift the complexity to the training data itself. We train the analog system to extract invariant representation vectors given two noisy views of the same input. This approach allows for learning locally and on the analog system itself. We experimentally demonstrate our approach with a free-space optical vector-matrix multiplier. We successfully trained in-situ a linear optical layer followed by a digital non-linearity and a linear classifier achieve state of the art test accuracy on par with digital simulations on MNIST. Next, we show that our method allows to continuously train the optical neural network while we apply some perturbations to it (misalignment, blurring, ...) while methods such as digital pre-training or Physics-Aware training cannot succeed at this task.

The rapidly emerging Generative AI technology that generates artificial images of real objects demands the development of a Detective AI technology that offers an efficient solution to solve the problem of distinguishing AI generated from real images with high accuracy. This paper presents an approach that utilizes topological properties of the bit-plane binary images and detects image components to characterize their connectivity and adjacency relationships. The feature vectors that are generated using cross-correlation matrix between these connected components, associated with bit-planes, are used to construct a feature space. This feature space is used to develop an efficient RF classifier that classifies AI-generated and real images with high accuracy. Simulations also show the cross-correlation between the connected components of 8 bit-planes of the grayscale images can help distinguish AI generated images from real images.

This work introduces a geometric deep-learning framework for the analysis of Single-Molecule Localization Microscopy (SMLM) point cloud data. The framework leverages a Graph Neural Network (GNN) to exploit the topological information inherent in point clouds, enhancing the detection, segmentation, and characterization of biological nanostructures in SMLM data. We demonstrate that our approach enables efficient clustering and topological analysis across different density conditions using minimal computational resources. We establish the reliability of our method by applying it to simulated and real SMLM datasets, covering a wide range of cluster structures and densities. Our results indicate that our approach outperforms the state-of-the-art clustering algorithms, especially in complex molecular organization scenarios.

A foundational component of vision processing is edge and feature detection. In the human eye, this is carried out powerfully via retinal ganglion cells which fight to suppress neuronal firing of neighbouring cells - a process termed 'lateral inhibition'. Such spatially-distributed activity competition leads to strong nonlinear enhancement of key image features such as edges, enabling more complex vision functionality including object recognition & motion detection. Software convolutional neural networks draw inspiration from this process and also begin with edge-detection, but in software this functionality is slow & the process intrinsically linear (matrix multiplications between the input image & convolutional kernels), with nonlinearity forced in via subsequent computationally-expensive activation functions such as ReLu. Here, we present a physical system which reproduces the strongly nonlinear lateral inhibition used in the retina. Using spatially-distributed mode-competition in nanoscale random network lasers, we demonstrate cutting-edge feature detection on complex images, and leverage this for a retinomorphic photonic convolutional neural network with strong performance.

Computation exploiting physics can provide outstanding energy efficiency, but physical systems tend to suffer from noise and unpredictability. This is for example the case of emerging memory devices. They can be used as artificial synapses for low-energy AI, but they suffer from variability, making them functionally analogous to random variables. In machine learning, Bayesian approaches are designed to operate with random variables. In this talk, we show that they can be an excellent way to exploit emerging memory devices without suffering from their drawbacks. We present three experimental realizations of Bayesian systems exploiting emerging memory devices: a Bayesian reasoning machine, which provides explainable decision-making, a Bayesian neural network, capable of quantifying the certainty of its predictions, and a Bayesian learning system. As these systems fully embrace, and sometimes exploit, device variability, they show outstanding robustness. We finally discuss the potential of Bayesian approaches to exploit other types of physics.

Ever since the first microscope in the late 16th century, scientists have been inventing new types of microscopes for various tasks, including phase contrast (1930s), confocal (1950s), light-sheet (1990s), and structured illumination (2000s). Inventing a novel architecture demands years, if not decades, of scientific experience and fortuitous creativity. In this talk, we try to invert the design thinking to supplement this creative process using data driven design approach we call Differentiable Microscopy.

To counter the exponential growth of computing power requirements for machine learning, efficient sailing of the integrated photonic processors has become a fundamental issue to be addressed. However, it remains challenging to properly calibrate the circuit imperfections, such as fabrication errors and crosstalk originating from both thermal and electric effects, which drastically affects the performance as the circuit size becomes larger. We demonstrate a silicon-photonics 16×16 Clements-type photonic vector-matrix multiplier. The degradation of fidelity caused by crosstalk and fabrication error was successfully compensated using our proposed machine learning based tuning method and deterministic calibration. The first experimental 10-digit MNIST classification was performed, which defines the classification results directly corresponding to the optical output ports. Furthermore, we also fabricated an 8  8 MZI-mesh photonic processor based on the planar lightwave circuit (PLC) technique which can realize low wavelength dependence operation due to low fabrication errors. This structure achieves the efficient throughput due to the O(N2W) operation, where N and W denote the number of spatial and wavelength channels, respectively. A high fidelity of 0.99 at 1550 nm and >0.96 over the C band was achieved, demonstrating the feasibility of the matrix-matrix multiplication operation with a combination of MZI-mesh and WDM.