Spectral Imaging in the Clinic

Ability to unmix data accelerates adoption of spectral imaging in biomedical applications.
29 June 2012
Richard Levenson and Alexandre Y. Fong

Spectral imaging, also known as multispectral or hyperspectral imaging, is gaining traction in research and clinical applications, thanks to improved hardware and software and the availability of multispectral reagents, such as quantum dots, and labeled primary antibodies.

Spectral imaging can offer optimized detection, validation, separation, and quantitation in several biological and biomedical applications. Its potential to both observe and quantify multiparameter events at the molecular and cellular level may provide insights into the basic mechanisms of life and deliver valuable diagnostic or prognostic capabilities for patient care.

Recent technological advancements have led to fast and accurate techniques for simultaneously probing cells and tissue for multiple molecules, retaining spatial information, and cleanly unmixing the data.

Spectral unmixing software can separate each overlapping signal quantitatively into its own channel—without crosstalk—and help eliminate confounding effects such as unwanted sample autofluorescence. Also, by having the spectral characteristics associated with each channel, a user can easily distinguish autofluorescent tissue elements from that of a dye and more accurately quantify the optical signals.

It is this ability to cleanly unmix signals virtually in real time that holds the most promise for further accelerating the adoption of spectral imaging in biomedical applications, including personalized medicine.

Many spectral imaging technologies

Spectral imaging has many different variants of technology and devices, including filtered cameras, whiskbroom and pushbroom scanners, multispectral laser confocal systems, Fourier-transform imaging spectrometers, computed- tomography imaging spectrometers (CTIS), image-replicating imaging spectrometers (IRIS), coded aperture snapshot spectral imagers (CASSI), and spectral-image-slicing snapshot approaches — and that only covers emission-side spectroscopy. Illumination light can also be spectrally modulated to provide similar information.

Spectral imaging technologies fall into roughly three classes:

  • Techniques that require sample illumination or camera movement to build up an image
  • Snapshot methods in which all spectral and spatial information is collected in a single exposure
  • Band-sequential approaches, in which whole images from different spectral bands are sequentially acquired (as in Fig. 1 below).
 Figure 1: Image stack acquisition via a band-sequential approach is shown at right. At left is the color-image of a collection of fl uorescent beads, and the middle schematic shows images acquired sequentially at a series of wavelengths. After “stacking” the data, each pixel will have its own spectrum associated with it.

Regardless of the hardware or technology used, a spectral imager delivers 3D image sets (x, y, and wavelength intensity) that contain spectral information at every pixel.1

A typical color image and its human visual counterpart is simply a spectral dataset containing only three (red, green, blue) intensities per pixel. And while “true” spectral imaging can deliver spectra containing hundreds or even thousands of spectrally distinct intensities at each pixel, such imaging spectroscopy doesn’t have much practical use in biomedical applications.

Band-sequential imaging for unmixing data

Microscopic examination of cells or tissues through spectral imaging usually involves fluorescently labeled specimens. However, bright-field examination of chromogenically labeled (light-absorbing rather than light-emitting) samples is also a promising arena, particularly in pathology, a field that has shied away from fluorescence-based techniques until recently.

Whichever labeling approach is used, data that combines x, y, intensity, and wavelength information must be acquired to allow for clean and robust signal unmixing.

Unmixing is a mathematical approach for resolving mixtures of spectrally distinct species, typically by fitting various proportions of known spectra, sometimes called endmembers, to the measured signal. The endmember spectra can be previously identified or — with the right tools —determined directly from the dataset being analyzed.

One of the most conceptually straightforward methods of accomplishing this is band-sequential imaging, which involves the serial acquisition of complete full-field images one wavelength band at a time. This is done using a filter wheel containing multiple transmission filters (up to 20 in some systems) located in the optical train in front of the camera. The system takes a series of images while switching filters to create a spectral data stack, or “cube.” Large fields of view and/or high pixel resolution can be obtained with this approach, and flexibility comes by having a many-position filter wheel, or the ability to swap different filters with different bandpass properties.

Advantages of this type of imager are that the user is in control of the number of spectral bands and the dynamic range can be preserved, since different exposures can be employed at different wavelengths.

This is helpful if, for example, a DAPI nuclear stain in the blue is extremely bright compared to a low-intensity fluorescent label in the green or red regions. The ability to take a very short exposure at 420 nm (blue) and a long exposure at 680 nm (red) allows both signals to be acquired with good signal-to-noise ratio and without camera saturation.

Disadvantages of the filter-wheel method include its size and the possibility of small image shifts from vibration as different filters are rotated into the field of view or if the filters are mounted incorrectly.

Relatively slow switching speeds of the mechanical device during acquisition times can also make it difficult to capture complete spectral stacks quickly enough to be useful in imaging living and potentially moving specimens.

A variant of this filter-based approach synchronizes a rapidly spinning filter disk with a high-speed camera to acquire images in quick succession, but this has some drawbacks as well.

Real-time unmixing technologies

Electronically tunable filters represent a promising advance over mechanical filter wheels because no noise or vibration is generated during wavelength switching. The bandpasses are also extraordinarily stable compared to older-generation glass filters that can age or delaminate. Electronic filters can be randomly tuned, so that bands can be obtained in any order. This can be an advantage during data collection if some dyes need to be imaged first due to photostability issues.

Also, with some technologies, transmission properties can be electronically altered to give control over the bandwidths and thus manipulate the trade-off between spectral resolution (the narrower the bandwidth, the higher the resolution) and light-capture efficiency (optimized with a large bandwidth).

Overall light throughput of tunable filters is less than that of traditional interference filters, by at least a factor of 2, because typical tunable filters are polarization-sensitive and usually have lower out-of-band light rejection.

Two major tunable filter technologies are liquid crystal tunable filters (LCTFs) and acousto-optical tunable filters (AOTFs).2, 3

Both have been incorporated into complete imaging systems for use with conventional microscope C-mounts and are accompanied with highly capable software suites. LCTF-based systems are available from Cambridge Research & Instrumentation, now part of PerkinElmer, Meadowlark Optics, and other companies. Gooch & Housego and Brimrose are two companies offering AOTF-based systems.

These systems can acquire spectral datasets over similar wavelength ranges with different filter designs to address visible vs. near-infrared vs. mid-IR regions.

The main distinction between the two approaches is filter-tuning speed. An LCTF typically can tune from one wavelength to another in 50 to 75 milliseconds, whereas an AOTF can tune from band to band in about 50 microseconds.

When coupled with a fast camera, the AOTF’s tuning speed opens up the possibility of near-real-time (i.e. video-rate) imaging in which complete spectral cubes (with a relatively small number of wavelengths) can be acquired, processed, and displayed without significant visual lag.

Classification and unmixing

Classification, the task of assigning an identity to different pixels, is often used in remote sensing and in military applications when it’s important to know whether or not something is a tank or other threat. However, the more common approach in biomedical applications is spectral unmixing, which treats pixels (or objects) as potential mixtures of signals and then determines how much of each signal in a multiple-stained sample is present at each location. The resulting calculated, unmixed images will display the spatial location and intensity of every label and autofluorescence, if present (Fig. 2).

 Figure 2: Spectral unmixing and removal of autofluorescence in tissue, DAPI and QD585 staining.

In the field of personalized medicine, for instance, this type of detailed molecular phenotyping is important in refining prognostic and therapeutic precision.4, 5

Software tools are necessary to take advantage of spectral imaging. The matrix math involved is straightforward, and the primary challenge is coming up with the correct estimates of the measured “pure” spectra of each signal. Once those are determined, the unmixing operation can be performed in milliseconds.

High-speed algorithms take advantage of the speed of AOTF technology and have yielded high frame rate (> 20 fps), or near- real-time processing of multispectral images. Quantitation of area and/or intensity values can then be achieved by applying standard image processing and segmentation steps to the unmixed images, even when there is colocalization of the various probes.

Multiplex labeling accelerates adoption

The commercialization of quantum-dot labels for spectral imaging means they can now be used with either antibodies or molecular probes, and there is a greater availability of primary antibodies directly labeled with a variety of fluorescent dyes.

Multiple chromogens, of course, exist for immunohistochemical staining. Fortunately, the common combination of DAB (brown), Fast Red (or the equivalent) for the molecular stains, and hematoxylin as the counterstain is well-suited for spectral unmixing (Fig. 3 below).

 Figure 3: Multiplexed chromogenic signals, brightfield display. Breast tissue was stained for the presence of estrogen receptor (ER/brown) and progesterone receptor (PR/red) and counterstained with hematoxylin. The ER and PR signals were spectrally unmixed, the PR signal pseudocolored green, and various combinations of these unmixed signals overlaid on original image. Co-expression of ER and PR can be seen as yellow signals in the top middle panel.

Getting past two or three chromogenic stains can be a challenge, but certain combinations of chromogens have been identified as being suitable for multiplexed spectral detection,6 and new chromogens are being developed with this application specifically in mind. This will greatly enhance the use of simultaneous molecular stains for research and clinical pathology.

When unmixing brightfield, chromogenically labeled samples, it is possible to display the unmixed results as in Fig. 3, in which the unmixed signals are layered over a brightfield background of just the hematoxylin signal to mimic the appearance of conventional single-color stains—or to re-color stains (in this case brown to green) to enhance legibility.

On the other hand, inverting the color space and presenting the unmixing results in “pseudo-fluorescence” can make some biological events easier to see by eye. In that case, it’s important to note that these are just display options and do not affect the underlying quantitative data.

Editor’s note: This article is adapted, with permission, from “Spectral Imaging for Bioscience Applications: Band-Sequential Techniques,” by Richard Levenson et al., which appeared in the February 2012 Photonics Online Newsletter.


1. J. M. Lerner et al. “Approaches to spectral imaging hardware,” Current Protocols in Cytometry Unit 12.20 (2010)

2. R. M. Levenson et al. “Spectral imaging in biomedicine: a selective overview,” Proceedings of SPIE 3438, 300-312 (1998)

3. E. S. Wachman et al. “AOTF microscope for imaging with increased speed and spectral versatility,” Biophysical Journal 73, 1215-1222 (1997)

4. R. M. Levenson et al. “Multispectral imaging and pathology: seeing and doing more,” Expert Opinion in Medical Diagnostics 2, 1067-1081 (2008)

5. C. M. Gilbert and A. Parwani. “The use of multispectral imaging to distinguish reactive urothelium from neoplastic urothelium,” Journal of Pathology Informatics 1(23) (2010)

6. C. M. van der Loos. “Multiple immunoenzyme staining: methods and visualizations for the observation with spectral imaging,” Journal of Histochemistry & Cytochemistry 56(4), 313-328 (2008)

SPIE member Richard Levenson is professor and vice chair for strategic technologies at University of California, Davis (USA), and president of Brighton Consulting Group. He has an MD from University of Michigan and previously served as vice president of Cambridge Research & Instrumentation.


SPIE member Alexandre Y. Fong is senior vice president of Life Sciences and Instrumentation at Gooch & Housego and president of the Florida Photonics Cluster. He holds a BS and MSc in experimental physics from York University (Canada) and an MBA from University of Florida (USA). Fong is a chartered engineer.


Quantum dots for hyperspectral imaging Quantum Dots

The availability of quantum dots has helped accelerate the adoption of spectral imaging in biomedical applications.

They were discovered by scientists at the Vavilov State Optical Institute (Russia) and later developed by Bell Labs (USA) in the 1980s.

They have been commercialized recently for pre-clinical and clinical applications as biological labels.

The two major virtues of these luminescent nanoparticles are that they are very photostable and have variable Stokes shift; many different colors can be excited by a single wavelength band in the near UV, which simplifies optical system requirements.

Primary antibodies labeled with various fluorescent dyes provide for simple labeling procedures. Yet, having these does not overcome the need for multiple-excitation wavelengths to cover the spectral gamut. 

New book takes holistic approach to hyperspectral remote sensing


"Hyperspectral remote sensing is a highly multidisciplinary field," writes SPIE Fellow Michael Eismann of Air Force Research Lab (USA) in the preface to Hyperspectral Remote Sensing, a new book from SPIE Press. "I believe that a student of this subject matter should appreciate and understand all of its major facets, including material spectroscopy, radiative transfer, imaging spectrometry, and hyperspectral data processing."

Cover of Hyperspectral Remote SensingMany resources cover these areas individually and focus on specific aspects of the hyperspectral remote sensing field. In this book, Eismann provides a holistic treatment that captures the field's multidisciplinary nature.

Hyperspectral Remote Sensing, available in hard cover and as an e-book, covers material spectral properties, the design of hyperspectral systems, and the analysis of hyperspectral imagery.

The content is oriented toward the physical principles of hyperspectral remote sensing as opposed to applications of hyperspectral technology.

More information: spie.org/PM210.

Have a question or comment about this article? Write to us at spieprofessional@spie.org.


To receive a print copy of SPIE Professional, the SPIE member magazine, become an SPIE member.


Recent News
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?