Applications of Machine Learning 2023

Until recently, evaluating the quality of unsupervised learning was too slow and expensive. This was a major hurdle to edge-enabled AI and any situations for which computational expense is a significant requirement. Adaptive Resonance Theory has been part of the solution because it can self-correct based on unsupervised category mismatch detection and reset. This advantage can be further leveraged by the development of incremental cluster validity indices. Validity indices provide various quality measures for unsupervised learning. Converting these to incremental versions is an approach that dominates prior methods, particularly for real-time or edge computing applications. Integrating incremental measures into the machine learning architecture further enhances these cost and speed advantages.

We present novel physics-guided neural network architectures for hyperspectral target identification. Specifically, our neural networks operate on at-sensor airborne hyperspectral radiance data to predict not only the material class, but also physically-meaningful quantities of interest, such as the atmospheric transmission factor, the temperature, and the underlying material emissivity. In this way, our models are decoupled from traditional preprocessing routines and provide independently verifiable and interpretable quantities alongside the class predictions. We compare our physics-guided models to more traditional black-box models with respect to classification accuracy and robustness to out-of-distribution data, and assess accuracy and consistency of predicted physical quantities.

We have developed a system applying deep learning super resolution (SR) to multispectral and hyperspectral geospatial satellite imagery to deduce higher resolution images from lower resolution images while maintaining original color of the lower resolution pixels. A super resolution model, which uses deep convolution neural networks (DCNNs), is trained using individual image bands, a large crop size or tile size of 512 x 512 pixels, and a de-noise algorithm. Applying our algorithms to maintain the original color of the image bands improves the quality metrics of the super resolution images as measured by PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index measure) of super resolution images. One of the most important applications of satellite images is to automatically detect small objects such as vehicles and small boats. With super resolution images generated by our system, the object detection accuracy (recall and precision) has improved by 20% with Planet Labs multispectral satellite images.

Elephant seals are no longer endangered but to continue to protect this success story, it is important to have an accurate count of how many remain in the wild. However, it is not an easy task to count their large and very dense groups. This research proposes an automated method of counting. A Grounded Language-Image Pre-training model was trained from drone images. The model produced bounding boxes of each seal’s location and a GMM classified males, females, and pups. The results compare favorably to human counting. This research will make it easier to count seals and track the population.

Machine learning applied to hyperspectral imagery can be used for gas plume detection and identification. In practice there are many difficulties, one of which is being able to estimate the background spectrum in order to increase differentiability of gases in the identification stage. We demonstrate that using image segmentation to locally estimate a mean background spectrum increased model prediction confidence by up to 20%. Furthermore, an ensemble of watershed segmentations can greatly reduce variability in model confidence.

Recent advances in deep learning, cloud computing, and Earth observation datasets enable a breakthrough in automated mapping and monitoring at global scale in near real time. We report on work generating a sequence of annual, global land use and land cover maps at 10m spatial resolution for years 2017 through 2022, publicly available as an open science product. Each map required processing over 2 million Sentinel-2 scenes (0.6 petabytes). Each map was completed in approximately one week using a cloud computing system. We report our map accuracy and recent work to improve the maps for detecting changes across years.

Robustness to image quality degradations is critical for developing deep neural networks for real-world image classification. Prior work explored how various optical aberrations degrade image classification performance. This paper extends this discussion to include optical scatter, which is fundamental to the stray light control of imaging systems and enables further discussion of DNN performance in the context of hardware design. In this paper, multiple state-of-the-art DNN models are evaluated for their image classification performance with imagery that has been degraded by optical scatter.

Convolutional neural networks yield activations from spatial frequency content in an image, allowing them to learn and recognize features of classification targets. This paper explores the spatial frequency response of CNNs in the context of an imaging system's modulation transfer function. Deriving the relationship between CNN design and imaging system design is a fundamental first step in optimizing these systems at the system level.

Images captured in smoky environments are corrupted. This affects monitoring infrastructures, and hinders emergency services during critical situations. In response, the Institute for the Protection of Terrestrial Infrastructures of the German Aerospace Center has worked to improve the security and safety of critical infrastructures. The proposed work uses deep learning models to enhance images of gauges captured in hazy and smoky environments. The study utilizes two deep learning architectures to improve the visibility of gauge images that are corrupted with smoke, where a new synthetic dataset was generated. The work shows that the use of deep learning improves the reading of gauges in smoky environments.

Attention mechanisms are one of the most prominent but least frequently used deep learning architectures in photonics. In this work, we numerically investigate their potential to be employed in both time- and frequency-domain applications. Briefly, we employ various types of neural networks to predict the coupling quality factor of microring resonators. Our results show that attention mechanisms are the most successful ones in terms of accuracy and learning efficiency enabling a six-times reduction in computing time.

Pixelated detectors placed on the board of modern space X-ray telescopes offer simultaneous imaging and spectroscopic capabilities, allowing one to recover properties of single photons coming from astrophysical sources. Despite the great performance characteristics of these detectors, they suffer from significant background induced by cosmic-ray charged particles. This background component greatly complicates the study of low surface brightness objects, such as the outskirts of galaxy clusters and very distanced active galaxies. In this work, we present the implementation of a U-NET-based image segmentation model which was developed for future X-ray detectors for space-based observatories.

We developed an approach to search for Ca II quasar absorption lines (QALs) using deep learning. We created large amount of simulation data as the training set and used an existing Ca II QAL catalog as the test set. Our solution achieved an accuracy of 96% on the test dataset and runs thousands of times faster than traditional methods. Our trained neural network model was applied to quasar spectra from the Sloan Digital Sky Survey’s Data Releases 7, 12, and 14, and discovered 542 brand-new Ca II QALs, the largest catalog ever discovered, which will play a significant role in creating new theories and confirming existing theories about galaxy evolution.

We report a generalizable computational approach to dramatically reduce biomolecular and chemical sensor response time for applications including medical diagnostics. Using experimental data to train an ensemble of long short-term memory (LSTM) networks, accurate predictions of equilibrium sensor response and associated uncertainty can be achieved from data measured over a short time span. This approach is particularly advantageous for sensor platforms with long response times due to poor mass transport, including porous silicon optical biosensors, which we use to validate this methodology through exposure to various concentrations of protein solution and subsequent analysis.

Modern face ID systems are often plagued with loss of privacy. To address this, some face ID systems incorporate image transformations in the detection pipeline. In particular, we consider transforms that convert human face images to non-face images (such as landscape images) to mask sensitive and bias-prone facial features and preserve privacy, while maintaining identifiability. We propose two metrics that study the effectiveness of face image transformations used in privacy-preserving face ID systems. These metrics measure the invertibility of the transformations to ensure the meta-data of the face (e.g. race, sex, age, etc.) cannot be inferred from the transformed image.

Using vision-text modalities to automatically perform visual or textual classification has attracted popularity in several applications. However, typical tasks require a large amount of completely aligned multimodal data, which are usually labor-intensive. In this paper, we propose a KD-CLIP to address the missing modality issue in classification by transferring the knowledge from the multimodal model to the monomodal model. We evaluated our method using a public dataset comprised of 11,788 image-text pairs for bird species classification. Our method achieves an overall accuracy of 86.42% on the image model, and 85.14% on the text model using a 5-fold cross-validation.

Surround-view fisheye cameras are commonly used for near-field sensing in automated driving such as urban driving and auto valet parking. Based on surround view cameras, there is much work on parking slot detection and classification mainly focused on the occupancy status in recent years, but little work on whether the free slot is compatible for the mission of the ego vehicle or not. For example, some spots are handicap or electric vehicles accessible only. In this paper, we tackle parking spot classification based on the surround view camera system. We adapt the object detection neural network YOLOv4 with a novel polygon bounding box model that is well-suited for various shaped parking spaces, such as slanted parking slots. To the best of our knowledge, we present the first detailed study on parking spot detection and classification on fisheye cameras for autonomous parking scenarios. The results prove that our proposed classification approach is effective to distinguish between regular, electric vehicle, and handicap parking spots. The application can be used in any other cars equipped with surround view cameras.

For nearly twenty years, a multitude of Compressive Imaging (CI) techniques have been under development. Modern approaches to CI leverage the capabilities of Deep Learning (DL) tools in order to enhance both the sensing model and the reconstruction algorithm. Unfortunately, most of these DL-based CI methods have been developed by simulating the sensing process while overlooking limitations associated with the optical realization of the optimized sensing model. This article presents an outline of the foremost DL-based CI methods from a practitioner's standpoint. We conduct a comparative analysis of their performances, with a particular emphasis on practical considerations like the feasibility of the sensing matrices and resistance to noise in measurements.

Several deep Convolutional Neural Networks (CNN) were constructed using daytime imagery data collected at the TISTEF laser range to predict the level of atmospheric turbulence. The shapley values computed from the best performing model are presented, providing insight into how the model makes predictions, as opposed to a black-box approach. The predicted values were very correlated with measurements taken from a Scintec BLS 900.

Achieving carbon neutrality has become the United Nation’s most urgent mission, but the lack of data, evaluation criteria and associated techniques presents a challenge. Moreover, the energy crisis in 2022 has unexpectedly complicated carbon dioxide (CO2) data, and existing research focuses primarily on CO2 absolute emissions. Policymakers have established milestones on carbon reduction roadmap but have failed to meet them. Therefore, we adopt the new CO2 emission and sink data released in November 2022. Our approach leverages Time Varying Parameter Vector Auto Regression (TVP-VAR) model and Monte-Carlo simulation to monitor the dynamics of net-zero emission roadmap. This approach provides insights into the global pathway towards The United Nations Framework Convention on Climate Change (UNFCCC).

This study proposes a depth estimation algorithm for picking tea buds in real environments using a novel light field imaging device. A Transformer is used to model the global relationship of sub-aperture images, obtaining feature information across sub-aperture images. An edge thinning Transformer network is proposed to restore edge details of tea buds. The use of dynamic modulation technology of occlusion pixels addresses occlusion problems. Experimental results show that this method is highly competitive with proposed light field depth estimation algorithms and has potential applications in the field of tea picking.

Detection of the source of the fault is an important issue in industrial products. According to the analyses regarding refrigerators, it has been determined that the majority of customer complaints are caused by noise-based complaints. Therefore, it is very important to identify the main source causing the noise problem and correct it as fast as possible. The aim of this study is to classify fan-related faults in refrigerators using sound signals. The method applied to diagnose the source causing the fault was preferred to be data-based, and for this reason, it was aimed to carry out the study with the help of a suitable algorithm that learns from the dataset. Creating a reliable and detailed dataset in order to improve the data infrastructure for use in this study and future studies is the secondary aim of the study. In this study, in the case of only one of the 3 fan sources of the refrigerator is faulty and all of them are working properly, a sound dataset is created by acquiring sound data in ISO 3745 compliant full anechoic measurement environment. An ensemble classification model is proposed by using a machine learning model trained by extracting the statistical features of the sound signal and a CNN (Convolutional Neural Network) architecture trained using mel spectrograms, which are the visual representation of the sound signal. The proposed model classifies with an accuracy of 93% when the non-faulty class is not included and 89% when the non-faulty class is included.

Convolutional Neural Networks (CNNs) are widely used for image classification and recognition tasks, but their computational complexity can be a challenge when deploying them on resource-limited edge devices. Sparse CNNs that use the Compressed Sparse Row (CSR) format can reduce memory footprint and computations while maintaining accuracy. This work proposes a novel approach for accelerating Sparse CNNs on Field-Programmable Gate Arrays (FPGAs) using the CSR format and systolic arrays. This approach outperforms a state-of-the-art GPU implementation in terms of computation time and resource utilization, making it a promising solution for accelerating Sparse CNNs on resource-limited devices.

Model-based machine learning methods incorporate domain knowledge from the physical forward model of an inverse problem to reduce the need for training data. In this research, we show how this can be used to address challenging limitations such as occlusion. We combine a convolutional neural network with a novel computational reconstruction method that combines source and attenuation distributions in order to model occlusion. We demonstrate the ability to quickly learn to address reconstruction artifacts and opacity, forming a significantly improved final image of the scene based on as little as a single training image. The algorithm can be implemented efficiently and scaled to large problem sizes.

The manual search for animal movement footage is a time-consuming task for biologists. AI and hardware devices like microcontrollers with cameras can help. The study explores automation of real-time animal identification using a night vision camera and a microcontroller. A motion detection algorithm triggers the camera and sends the image to an image classification model in the cloud through REST API. Predictions are displayed on the device's LCD screen. Two deep learning models, MobileNetV2 and ResNet50, were constructed and evaluated for accuracy. The ResNet50 model produced the best results and was deployed as a REST API service.

We use unsupervised machine learning approaches in a completely data-driven binarization routine for vibration sensors with minimal lag. Gyroscopic vibration sensors are inherently noisy as they report analog signals that must be translated into digital values, in our case whether a pump is running (“on”) or not (“off”). When analyzing data from different pumps, each of which has their own baseline vibration values and magnitude of vibration, manual annotation is not feasible. We have tested multiple unsupervised methods including k-means clustering, Gaussian naïve Bayes, and ensemble learning to correctly binarize the analog signals. Comparisons are made on the basis of “blips” or times where the algorithm predicted an incorrect state for a short period of time before returning to the current state. This provides an objective metric with which to evaluate an algorithm’s success at binarizing the signal. We present results from an experimental design to probe the efficacy of different learning methods across data collected from ten vibration sensors deployed on pumps in a water treatment plant. We use initial k-means clustering for many algorithms to get an initial guess of the on or off state of the pump. From there we use a variety of smoothing, Gaussian naïve Bayes classifiers, and ensemble learning to get a final classification of pump activity. We apply these methods to data collected from ten sensors deployed on distinct pumps.

Low-light image enhancement has become a significant challenge in recent years due to non-uniform luminance in images. Real-world images often have non-uniform luminance, requiring local enhancement in certain areas. To address this issue, this paper presents a new methodology utilizing two convolutional network architectures. According to the results obtained, the proposed method outperforms algorithms with more complex architectures in the literature. The double convolutional architecture emphasizes local enhancement in real-world scenes and global enhancement in images with very low luminosity. Overall, this paper presents a significant contribution to low-light image enhancement offering an effective solution to the non-uniform luminance images.

This study compares the performance of features extracted with Histograms of Oriented Gradients (HOG) and with Convolutional Neural Networks (CNN) in classifying normal and keratoconic eyes using anterior corneal surface maps. Both models achieved high accuracy: HOG perfomed 99.1% sensitivity, 98.7% specificity, and AUC of 0.999143, while CNN had 99.5% sensitivity, 99.5% specificity, and AUC of 0.999778. Results suggest comparable performance for both feature types.

With the democratization of UAS (Unmanned Aircraft Systems) from defense and security to entertainment, drones need to perform automated navigation in a broader range of illumination scenario. This implies improved autonomous navigation through depth estimation and features such as long-distance object identification. We present our lightweight ultra wide-angle camera optimized for low-light illumination (down to 2.5 lux) mounted on a drone using a dual camera to capture stereo images. We then compare our module with other drone cameras on neural networks for in-flight depth estimation and object identification to highlight the benefits of our camera for traditional drone applications.

Prescribed fires are an important part of forest stewardship in Western North America, understanding prescribed burn behavior is important because if done incorrectly can result in unintended burned land as well as harm to humans and the environment. We looked at ensemble datasets from QUIC-Fire, a fire-atmospheric modeling tool, and compared various machine learning models effectiveness at predicting outcome variables, such as area burned inside and outside the control boundary, and if the fire behavior was safe or unsafe. It was found that out of the tested machine learning models random forest performed best at predicting all three predictor variables of interest.

In this work, we utilized deep learning models for depth image regression to predict chicken weights. The dataset consists of annotated 99,427 depth images obtained from 18,706 chickens standing on the weighing scale during the rearing days 21-84. Pretrained models performed regression on the depth image data, including MobileNet V2, ResNet50 V2, ResNet101 V2, ResNet152 V2, InceptionV3, and Xception. All models performed comparable results regarding mean absolute error (MAE) and mean relative error (MRE); however, Xception performed best with an MAE of 17.2 g and an MRE of 2.52% on the test dataset compared to the reference weight. Based on these results, chicken weight estimation using depth images and deep learning is a promising technique for daily growth rate monitoring for the poultry industry.

This paper presents a frequency-driven deep learning approach as an alternative to the convolutional neural network (CNN) that generally requires many images that have high discriminant spatial and longitudinal features, within and between classes for comprehensive learning. It first presents a new backyard birds’ dataset that is extracted from RGB videos and consisted of the images of cardinal and sparrow images and uses it to develop an artificial neural network (ANN) model with the Kaiser–Bessel window and the fast Fourier transform (FFT). Simulations show that the ANN model can predict the feature vectors of the birds with high-performance scores.

Plasmonic metamaterials providing enhanced local optical chirality have received significant attention for ultrasensitive molecular chirality sensing. More rigorously, analytes cause chiral energy redistribution in the plasmonic resonance, and its spectroscopic measurement enables quantitative analysis. Herein, we suggest an algorithmic strategy to target-specifically optimize the energy redistribution and spectral sensitivity of nanostructure. Chiroptical properties of the plasmonic Bonn-Kuhn model was studied with resource-efficient neural networks and were combined into a genetic algorithm to efficiently derive an optimal structure for a specific analyte. The spectral sensitivity was tailored to the maximum by the algorithm, and optimization of chiral energy redistribution was achieved.

In this paper, we create mix-and-matched generative networks to address privacy and bias concerns in face recognition systems. There has been a rise in bias based on religion, gender, and race. To preserve the robustness of face ID systems while masking these bias-inducing facial features, we map the faces to neutral natural landscape images. This still leaves the possibility of estimating facial features from the landscape images. We address this issue through decorrelation shuffling functions between the latent spaces of the encoder and the generator networks, as a way of decorrelating facial and landscape features and preventing hacking.

Retinopathy is a common complication of diabetes that can cause severe vision loss if not detected and managed promptly. In this study, we propose a comprehensive approach that leverages image processing techniques to analyze fundus images of patients with diabetic retinopathy. Our primary focus is on vein extraction and hemorrhage detection, with exudate detection being performed only on specific images to showcase advancements in the current prototype algorithm. The dataset used in this project consists of images obtained from Mexican ophthalmology institutes, ensuring its relevance and applicability to the local population. By extracting veins and hemorrhages, we aim to capture crucial features indicative of the severity of retinopathy. These generated images, along with the original dataset, are utilized to train convolutional neural network (CNN) models, enabling accurate classification of the disease's degree into three categories. The significance of this project lies in its potential to serve as an auxiliary tool in diagnosing diabetic retinopathy. By automating the analysis of fundus images and providing objective classification results, our algorithm aims to assist health

Sparse coding has long been thought of as a model of the biological visual system, yet previous approaches have not been used to model the activity of individual neurons in response to arbitrary images. We present a novel model of primary cortical neurons based on a biologically-plausible sparse coding model termed the locally-competitive algorithm (LCA). Our LCA model is trained on a self-supervised objective using a standard image dataset and a supervised objective using a modern neurophysiological dataset. Our novel sparse coding model better represents the computations performed by biological neurons and is significantly more interpretable than previous models.

In Bioinformatics, batch effect detection is a challenging task where the clustering approaches have been explored most of the time. In this study, we proposed a novel approach to identify batch effects and visualization with unsupervised analysis methods. We used the most significant gene sets 500,1500, and 2500 genes out of 35238 genes for the human-liver RNA seq dataset by applying standard deviation (SD). The skmeans and kmeans methods were explored on the selected gene subsets. Then, principal component analysis (PCA) was used for embedding to the 10-dimensional subspace. Finally, the Uniform Manifold Approximation and Project (UMAP) was applied to cluster and visualize the outputs. The experimental results demonstrate the robust representation and achieve the best clustering and visualization for features extracted from 1500 genes. These findings are not only useful for batch effect detection and removal tasks but also can be used to label new samples to train the supervised machine learning methods.

Deep learning and machine learning architectures have proven to be effective for medical imaging segmentation. In this research, we examine the effectiveness of various deep learning methods, including CNN-based and transform-based approaches, for skin lesion segmentation. We make use of a publicly available dataset of dermoscopy images with ground truth segmentation masks and evaluate multiple approaches to skin lesion segmentation tasks. We demonstrate that the deep learning methods trained with polar transformations as a pre-processing step improves robustness and pixel-level recall while achieving better segmentation performance.

Alexithymia describes a psychological state where individuals struggle with feeling and expressing their emotions. Individuals with alexithymia express atypical attention to the eyes when recognizing emotions. Using a public data set of eye-tracking data from seventy individuals, we aim to train machine learning models for alexithymia classification. We train several models for alexithymia classification including support vector machines, logistic regression, decision trees, random forest, and multilayer perceptron. We use 10-fold cross validation to compare the performance of the models. We achieve 80.0% mean accuracy over 10 cross validation folds using the multilayer perceptron model.

Artificial intelligence (AI) based analysis accelerates clinical diagnosis from pathological images efficiently and accurately. Due to the high dimensionality of pathological images, extracting meaningful feature representations of the pixels from high-dimensional images is essential. This can be used for further analysis to obtain better insights. This study used Deep Convolutional Neural Networks (DCNN) and end-to-end Deep Convolutional auto-encoder outcomes (DCAE) models for feature extraction. K-means and K-Nearest Neighbors (KNN) methods were then used for clustering and classification and achieved 95% testing accuracy with these unsupervised classification methods. In addition, t-distributed Stochastic Non-linear Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) were applied for clustering and visualization of different tissue types and demonstrated promising representation for histopathological image clustering of 20 different tissue types. Within-Cluster and Square (WSS) errors were used to determine the optimal number of classes in cluster space with t-SNE and UMAP methods. Most importantly, the proposed system relies on class probability and the visual interpretation that provides the relationship among the 20 different pathological tissue types. The proposed pipeline is potentially applicable for understanding pathological image classification and clustering tasks to obtain better insight into digital pathology applications.

Unmanned aerial vehicles (UAVs) have enjoyed a meteoric rise in both capability and accessibility—a trend that shows no signs of slowing. This has led to a growing need for detect-and-avoid technologies. These increasingly commonplace events have resulted in the development of a number of UAV detection methods, most of which are based on either radar, acoustics, visual, passive radio-frequency, or lidar detection technology. With regards to software, many of these UAV detection systems have begun to implement machine learning (ML) as a means to improve detection and classification capabilities. In this work, we detail a new lidar and ML-based propeller rotation analysis and classification method using a wingbeat-modulation lidar system. This system has the potential to sense characteristics, such as propeller speed and pitch, that other systems struggle to detect. This paper is an exploration into the preliminary development of our method, and into its potential capabilities and limitations.

Model cards organize and portray information about a model’s training data, hyper parameters, and behavior, such as predictive accuracy and bias for machine learning (ML) algorithms or models. Consumers use these cards to determine whether a model is well suited for a particular use case. However, the current design of model cards does not include resource utilization metrics, which are important when models are being optimized for specific hardware. The main objective of this study was to determine which set of hardware-centric performance metrics provided the most value to users when comparing among different algorithms for edge devices.

We present a generic approach to assist assembly line workers in learning standard operating procedures (SOP) for product inspection. Our approach uses computer vision and machine learning on the edge. A video stream is analyzed using human and product pose estimators, and the resulting data input into an action classifier to determine the current inspection step. A graphical user interface provides real-time feedback on SOP adherence. Human operators can handle complex scenarios but often lack consistency. Our approach aims to minimize operator variability by leveraging automation as an assistive technology.