Hyperspectral Imaging and Applications II

In this study we present a hyperspectral imaging (HSI) system that comprises of two commercially available XIMEA xiSpec 2.2 MPixel snapshot mosaic cameras and focuses on the spectral window 470-920 nm. The system design meets the demands of static sampling and remote sensing when mounted on an Unmanned Aerial System (UAS). Details on the system’s set up – both in lab and on field, the calibration procedures and the image analysis will be given with focus on addressing major calibration challenges related to the nature of the HSI system presented in this study.

We present a novel hyperspectral imaging system working in the visible and in the short-wave infrared (SWIR) spectral region based on a Fourier-transform approach. The technology presents an exceptional light throughput, a high spatial resolution, a software adjustable spectral resolution, and a wide versatility of use. In this work, we illustrate a broad portfolio of applications both in the visible and in the SWIR regions, with particular focus on microscopy and biology, cultural heritage, and quality control for the agri-food sector, in collaboration with a vertical farm.

Hyperspectral imaging has over the last thirty years developed into a power analytical tool for the determination of chemical and other properties. As a result, there has been strong development in both the design of spectral cameras and in the applications for which they are used. This has led to a diversity in the way fundamental instrument performances are characterized, reported, and understood. As a result, this makes it difficult to compare instruments for application- specific needs, or for commercial market needs. In 2018, the IEEE P4001 group was formed to facilitate the development of a standard to unify the use of terminology, spectral camera characterization methods, and the meta-data structures that are needed to represent spectral camera performance. This talk provides an update on the work to date, and the significant progress made towards the first draft of the standard which is scheduled to be available for review at the end of year 2022.

There is a growing interest in SWIR to solve many problems across a range of applications. Major advances have been made in Hyperspectral Imaging over the past few years. The data collected in a HIS system for analysis relies on a good camera, offering a combination of resolution, speed and sensitivity. In this presentation Mark Donaghy, from Raptor will discuss the many parameters in selecting the right camera and will then go through a series of example applications that customers are working on using Raptor SWIR cameras. He will discuss where SWIR technology / HIS goes next.

Imec presents its latest snapshot hyperspectral imagers, based on a 3x3/4x4/5x5 mosaic pattern, in either VIS (460-630 nm), NIR (600-960 nm) and SWIR range (1100-1650 nm). These compact-form cameras offer either 9, 16 or 25 bands and a spatial resolution of 640x512 pixels (SWIR) or 2048x1088 pixels (VIS/NIR) with high acquisition speed of 120 cubes per second making it suitable for conveyor-belt inspection. This paper presents a wide range of applications where snapshot cameras in either VNIR or SWIR range have shown potential, covering applications in recycling, agriculture, industrial inspection, object tracking and particle identification.

HySpex is presenting an integrated solution for hyperspectral drill core imaging. The system’s mineral mapping capabilities are studied in close cooperation with renowned academic and industrial partners through the Center to Advance the Science of Exploration to Reclamation in Mining (CASERM). The system around HySpex cameras in the spectral range of 400-2500 nm scans full core boxes within seconds providing real-time mineral mapping using Prediktera’s Breeze Geo software. Among others, mapping via the publicly available USGS Material Identification and Classification Algorithm (MICA) is provided. Drill cores from the LaRonde-Penna deposit are evaluated. The volcanogenic massive sulfide (VMS) deposit, contains an endowment of 71 Mt of ore grading on average 3.9 g/t Au and economic grades of Zn, Cu and Pd, the cores exhibit hydrothermal alteration and regional metamorphism. Validation is provided by using the Minalyze CS continuous X-Ray Fluorescence (XRF) scanner and via previous peer-reviewed studies of the deposit.

The objective of this study was to establish a non-invasive and high-throughput grading system for cocoa bean using hyperspectral imaging combined with machine learning. Cocoa beans were scanned using a HySpex SWIR camera covering the spectral range from 970 to 2500 nm. A maximum entropy model was built to identify each bean into different classes: good beans, under-fermented beans, slaty beans, and other low-quality beans. The classification was validated by cut test. A classification accuracy close to 80% was achieved without having to cut the beans open, revealing the great potential of hyperspectral imaging to accurate quality control of agricultural products.

This work reviews and presents a comparison of hyperspectral imaging results when analysing corrosion products in the ultraviolet (UV) range (250 nm – 500 nm), visible near-infrared (VNIR) range (400 – 1000 nm) and shortwave-infrared range (900 – 2500 nm). In related and prior work in our group, corrosion products on steel have been detected using hyperspectral imaging in the VIS, NIR and SWIR regions of the spectrum. However, an extensive review of the academic literature has revealed that the hyperspectral response of corrosion in the UV has not been reported. To address this, we present our results of imaging corrosion products on metal substrates using our Headwall UV-VIS Hyperspectral imaging sensor. These results are contrasted with the same samples imaged using our Headwall VNIR E series and Headwall SWIR 640 Hyperspectral imaging sensors. Our initial results indicate that corrosion spectra in the UV are separable from those of steel, but that the VNIR is the most appropriate range for this type of determination.

We describe results from experiments investigating how hyperspectral data might be incorporated into autonomous inspections for offshore turbines, part of Dr SUIT– (Drone Swarm for Unmanned Inspection of Wind Turbines), a collaboration funded by InnovateUK (UKRI). Imagery and point measurements were captured of small turbine blades subjected to damage by abrasion, impact and UV exposure. The technique appears effective at classifying abrasion damage to a degree comparable with conventional inspection schemes. Impact damage could be classified as ‘lower’ or ‘higher’ energies. The blades designed resilience to UV meant that little change was detected in those tests.

The operation of hyperspectral imaging systems in industrial environments can be a challenge. In the nuclear industry, partially transparent elements such as gloveboxes or panels are often used to cover samples for protection against the risk of contamination. In practical terms, this means that the hyperspectral sensors can only capture data through partially transparent media, which interferes the vision between sensor and sample. Representative examples of these media are Polymethyl Methacrylate (PMMA) or acrylic and Polycarbonate (PC). In this work, we evaluate the effect that the transparent media can have on the data when captured under these conditions, where transparent materials are placed between sensor and sample. Experiments include hyperspectral images of the same samples captured with and without panel obstruction for a direct comparison of spectral responses, suggesting potential artificial intelligence techniques and methods to identify these effects and mitigate them.

Solar energy will be the largest single source of electricity in a low-carbon future. To maximise the potential of solar power, new materials and technologies will be needed to harvest and convert solar energy alongside the dominant, silicon-based photovoltaic technology. Molecular electronic materials, such as conjugated polymers and molecules, are appealing because of the potential to tune their properties through chemical design and their compatibility with high-throughput manufacture. Through a remarkable series of advances in materials design, the efficiency of photovoltaic energy conversion in molecular materials has risen from 1% to around 20% within two decades, surpassing most predictions. We will discuss the function and status of molecular solar cells as well as the challenges and opportunities for further development.

In the brain tumour resection surgery, functional brain mapping (FBM) has been adapted in the contemporary neurosurgical workflow for improved surgical outcome. It allows neurosurgeons to identify and preserve brain regions with critical functions, thus to prevent post-surgical neurological deficits. However, there is no effective ways to directly visualize brain functions in real-time intraoperatively. A compact functional brain imaging device that can be more easily utilised during surgery is highly desired. We’ve been developing a multispectral imaging system (MSI) with the aim to fill this unmet clinical need. The device has been designed to detect brain functional activity by characterising light reflectance changes due to the blood oxygenation/flow fluctuation and neuronal membrane change. Co-localised MSI and fMRI measurements have been performed for detecting brain response to external electrical stimulations on preclinical animal models. Results suggested MSI could be able to identify the stimulation-evoked brain region as validated by the fMRI findings

Hyperspectral or imaging spectroscopy is a unique technique by which spatio-spectral information can be parameterized most effectively. Our novel development provides the opportunity for both science and industry to find solutions for medical applications by putting the latest portable spectroscopy into imaging practice. Cubert GmbH (Germany) has recently developed a miniature (29x29x49mm, 120 g) snapshot hyperspectral imaging camera based on light field technology, where both the intensity and direction of incident light rays are used to produce 62500 spectra/image. The camera combines a continuously variable bandpass filter, a multi-lens array and a highspeed CMOS sensor. This spectral camera comes with a native resolution of 250x250 pixel, resulting in 3 million data points/cube with 50 spectral bands covering 450-850nm, taken in a single snapshot. The camera generates up to 24 full hyperspectral data cubes per second enabling the benefits of video spectroscopy. With an optimized software environment, real time image processing and classification of this high speed spatio-spectral dataset is enabled. Our snapshot imaging spectrometer captures the entire scene in a few milliseconds or less, which has significant impact in real-time spectral imaging. In our presentation, we will highlight unique applications made possible by our light field hyperspectral cameras. We will provide an insight into the latest development of early-stage Alzheimer disease detection using a hyperspectral camera. We will also illustrate the advances in upcoming endoscopic devices and the usability of hyperspectral cameras in microscopy.

This work presents the feasibility of supervised deep learning architectures, namely U-Net++ and DeepLab-V3+, to perform the automatic segmentation of fifteen intraoperative glioblastomas hyperspectral images retained from the HELICoiD database. The HELICoiD ML-based pipeline conveys the gold-standard procedure to retrieve a segmentation map of brain cancer, as it is the only feasible way to label medical data when ground truth is unavailable via pathology-confirmed testing. The goal is to differentiate glioblastoma from other brain tissues, researching ways to analyse the images end-to-end and improve the time required to process different supervised and unsupervised algorithms in a unique ML pipeline.

We propose a hyperspectral imaging blueprint, designed to work with pushbroom sensors, which only seize one scene line at a time, offering high spatial and spectral resolutions. It can work in any scenario involving a motionless subject. It is affordable, open-source and robust. The system comprises Python libraries, an Arduino One platform, a Nema17 stepper motor, its driver controller, and a recirculating ball screw for accurate movement. It offers a diode-based targeting system to centre the image capture and measure the right focusing distance. A graphical user interface lets physicians interact, move and acquire the diagnostic data from the camera.

Hyperspectral cameras can acquire highly useful data for geology, agriculture, urban planning, and many other applications. Several satellite-based hyperspectral cameras are currently operational. Even big instruments usually have relatively large ground sampling distance (GSD): ~10m in 400-1000nm range and ~30m in 900-2500nm range (even coarser in microsatellite cameras). Our approach has been to provide significantly improved GSD, small keystone and smile, while keeping acceptable spectral sampling and SNR. The instrument development has been funded by the Norwegian Space Agency. The Agency has selected one of the proposed instruments as the primary payload on an upcoming Norwegian In-Orbit Demonstrator satellite.

Remote sensing applications are nowadays widely spread in various industrial fields, such as mineral and water exploration, geo-structural mapping, and natural hazards analysis. These applications require that the performance of image processing tasks, such as segmentation, object detection, and classification, to be of high accuracy. This can be achieved with relative ease if the given image has high spatial resolution as well as high spectral resolution. However, due to sensor limitations, spatial and spectral resolutions have an inherently inverse relationship and cannot be achieved simultaneously. Hyperspectral Images (HSI) have high spectral resolution, but suffer from low spatial resolution, which hinders utilizing them to their full potential. One of the most widely used approaches to enhance spatial resolution is Single Image Super Resolution (SISR) techniques. In the recent years, Deep Convolutional Neural Networks (DCNNs) have been widely used for HSI enhancement, as they have shown superiority over other traditional methods. Nonetheless, researches still aspire to enhance HSI quality further while overcoming common challenges, such as spectral distortions. Research has shown that properties of natural images can be easily captured using complex numbers. However, this has not been thoroughly investigated from the perspective of HSI SISR. In this paper, we propose a variation of a Complex Valued Neural Network (CVNN) architecture for HSI spatial enhancement. The benefits of approaching the problem from a frequency domain perspective will be answered and the proposed network will be compared to its real counterpart and other state-of-the-art approaches. The evaluation and comparison will be recorded qualitatively by visual comparison, and quantitatively using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Spectral Angle Mapper (SAM). The project can be access at: https://github.com/NourO93/3D_CCVNN_Hyperspectral.

The mid-wave infrared (MWIR) spectrum contains an abundance of invaluable information. From thermal signatures to the spectral fingerprint of different chemical species, MWIR imaging and sensing has wide ranging applications from surveillance, remote sensing to anti-counterfeiting. Spectral discrimination can be achieved using several approaches including: multilayer interference filters integrated in a mechanical filter wheel ; Fabry-Perot-based micro-electro-mechanical systems (MEMS) filters; acousto-optic tenable filters (AOTFs); and focal plane array (FPA) filters. These approaches suffer from numerous drawbacks such as limited spectral bandwidth, slow response times and moving parts. A solution spectral filtering solution which provides electrically tunable narrowband filtering across the MWIR is highly desirable. In this work we introduce actively tunable optical bandpass filters operating in the MWIR using the phase-change material (PCM) GexSbyTez (GST). The GST exhibits a large, reversible change in its refractive index across the MWIR upon phase transition (amorphous to crystallization). The PCM is integrated into different filter designs including multilayer interference filters and plasmonic nanoholes, whereby the refractive index modulation governs the filter’s optical characteristics. Through an applied electric field—which changes the GST state from amorphous to crystalline—we actively tune the filter to provide a continuum of narrow passbands and show spectral imaging capability using the devices.