One of the most challenging target detection problems is that of a stationary target in a cluttered environment, a scenario in which the target-to-mean-background differential (or signal-to-clutter ratio (SCR)) is small compared to the background fluctuation. This is a baseline long wave IR (LWIR; 8 to 12 µm) scenario characteristic of well-camouflaged targets. Forward-looking IR (FLIR) imaging sensors try to overcome SCR challenges with limited success by using high-spatial-resolution focal plane arrays (FPAs). Broadband mid-wave IR (MWIR; 3 to 5 µm) and LWIR sensors are limited in their ability to detect camouflaged targets in the presence of significant clutter.
Another method to detect and identify targets in clutter and partial obscuration is by leveraging spectral data as well as spatial data. Multispectral or hyperspectral imagers (MSI/HSI) capture the spectral data of a scene in multiple, narrowly defined wavebands. Spectral imagers can use line sensors that operate in scanning or sweep mode, or FPAs that operate in staring mode (see oemagazine, March 2004, p. 18). All of these MSI/HSI imager types can be used in conjunction with statistically based detection algorithms that exploit the unique spectral characteristics of target objects compared to their background environments to substantially increase the SCR and the likelihood of automated target recognition on the battlefield. MSI/HSI systems are difficult to maintain and operate, however. By combining spectral detection with traditional FLIR imagery, we can greatly enhance overall performance. What designers require, therefore, is a system that represents a radical departure from current approachesone that combines the best attributes of broadband FLIR and traditional HSI on a single FPA chip. Unifying FLIR and HSI
In many ways, a MSI/HSI system is simply the next step in FLIR imaging evolution. MSI/HSI systems use various configurations to acquire detailed spectral data from a scene, including a variety of sensors placed behind dispersive components such as gratings or prisms, or Fourier-transform spectrometer designs. An MSI system with a UV, visible, and/or IR sensor(s) may image over several spectral bands; an HSI may image over hundreds. As a result, MSI/HSI systems produce enormous amounts of data. MSI/HSI data sets, typically referred to as data cubes, consist of a 2-D spatial image of the scene with a vertical wavelength axis.
Data cubes represent a huge computational burden, making it difficult to process and communicate MSI/HSI data in real-time tactical situations. Also, the time needed to acquire such rich spectral data means that MSI/HSI instruments can fail when trying to detect rapidly moving objects in relation to the MSI/HSI platform, as is often the case in airborne reconnaissance. In reconnaissance applications, a spectral imaging sensor operating in staring versus scanning mode, and with a select set of wavelength bands, is advantageous to avoid image blur and synchronization issues.
All of these factors lead to MSI/HSI system designs that are expensive, large, and heavy, making them difficult to deploy on small platforms, particularly unmanned aerial vehicles (UAVs). The U.S. Defense Advanced Research Projects Agency (DARPA) Adaptive Focal Plane Array (AFPA) program aims to change this. The overall goal of the AFPA program is to demonstrate a high-performance FPA that is widely tunable across the relevant wavebands in the IR spectrum, including the short-wave IR (SWIR; 1 to 2 µm), MWIR, and LWIR bands (see figure).
Figure B. The goal is to produce an image-sensor array in which the wavelength sensitivity of each pixel can be independently tuned. In effect, the device would constitute a large-format array of electronically programmable microspectrometers.
Although a fixed-band chip would be easier to achieve, identifying a set of unique spectral bands of universal interest to serve all applications has proven difficult. With continuous spectral tuning, system designers could preprogram the AFPA to focus on specific spectral bands critical to that mission. A limited number of bands for each mission would sharply reduce the amount of data to be processed, possibly allowing real-time video display of false-color multispectral imagery or immediate automatic target recognition. Lastly, by requiring that the FPA be electrically tunable on a pixel-by-pixel basis, operators can program the AFPA to maximize either spectral coverage or spatial resolution. From Concept to Reality
Today, an IR sensor with the above characteristics does not exist. The integration of various component technologies into an AFPA involves a complex interplay across a broad range of disciplines, involving micro-electro-mechanical systems (MEMS) device processing, optical coating technology, micro-lenses, optical system modeling, and FPA devices. The goal of this integration will be an adaptable FPA with:
- On-chip spectral interrogation
- No complex mechanical components external to FPA
- No cooled filters external to FPA
- Intra-scene spectral programmability
- Pixel-to-pixel tunability within a single frame (intra-frame) and frame-to-frame tunability (inter-frame)
- Sensitive to up to three IR bands: MWIR, FLIR combined with multispectral LWIR, extendable to include possible SWIR
- Moderate to high spectral resolution (dλ/λ below 1% (less than 100 nm at 10 µm))
- Moderate frame rates60 Hz with kHz sub-frame tuning for frame-to-frame temporal registration
- Automatic pixel registration
- Spatially registered, temporally simultaneous pixels
- Compact format
- ƒ/2 to ƒ/4 field-of-view
- Affordable cooling
We are conducting the program in two phases. Phase I is an 18-month effort that will concentrate on the development of individual pixel structures that permit spectral tuning within the relevant IR atmospheric windows, complete spectral coverage within a selected waveband, narrow full width half maximum, and high sensitivity. In Phase II, we switch the emphasis to the integration of pixels into large format FPAs. This phase will demonstrate simultaneous multi-waveband operation while optimizing the tradeoff between FLIR and MSI/HSI capabilities. The expected outcome of the third and final phase of the program is a high-performance electro-optical imaging sensor with the capability to exploit both the spectral and spatial content of a scene.
Rockwell Scientific Co. (RSC; Thousand Oaks, CA) is one of the major performers in the AFPA program. The RSC design permits simultaneous spectral tuning in the LWIR region while providing broadband FLIR imagery in the MWIR band. Their goal is to develop, demonstrate, and deliver an array of MEMS-based tunable Fabry-Perot cavity filters integrated with a HgCdTe-based FPA. The MEMS filter array will initially enable the tuning of pixel sub-arrays, then evolve to tunable individual pixels. The device will undoubtedly require a new read-out integrated circuit (ROIC) to accommodate the additional control functions at each pixel. Another DARPA program is working on the ROIC issues.
DRS Technologies Inc. (Parsippany, NJ) and University of Western Australia (UWA; Crawley, Australia) have teamed up to develop and demonstrate AFPA technology in the SWIR region of the spectrum by integrating a MEMS optical filter with a HgCdTe-based FPA. This collaborative effort will combine tunable IR-filter technology from the UWA, optimized for SWIR operation, with the DRS high-density, vertically-integrated photodiode (HDVIP) IR-detector technology for a form-fit replacement of existing FPA devices such as those used in modern FLIRs. DRS will provide SWIR HgCdTe detectors and develop SWIR HgCdTe avalanche photodiodes (APDs) for low-light operation. In addition, DRS will team with the University of Texas (Austin, TX) to model and establish the fundamental performance characteristics of the SWIR HgCdTe APD.
The initial Phase I DRS/UWA research will result in a proof-of-concept demonstration of an integrated tunable MEMS filter on a detector test bar, followed by a Phase II effort to develop and demonstrate an integrated FPA with a single tunable mirror. Phase III involves the demonstration of a full-scale integrated 64 X 64 SWIR FPA with individual tunable filters for each pixel.
Electro-optical imaging sensors fulfill a wide variety of difficult military reconnaissance, surveillance, and targeting needs. New mission requirements, defined by UAV and robotic reconnaissance applications, are driving the need for smaller, lightweight image sensors with more capability than the current generation. These systems must obtain information from difficult-to-access areas and address obscured and camouflaged targets. The imaging sensors must also continue to function in difficult environmental and atmospheric conditions.
A capability to detect spatial information in multiple spectral bands at the focal-plane level will enhance electro-optical capability beyond that of conventional FLIR systems. Thermal contrast reversals, camouflage matched to a particular background, and the variety of environmental conditions encountered worldwide present significant challenges to single-band sensors. Without a means of multispectral band discrimination, there exists significant risk that targets will be missed and threats to both people and vehicles not detected.
The realization of the AFPA concept offers the potential for dramatic improvements in critical military missions involving day/night reconnaissance, battlefield surveillance, and precision targeting. Owing to its adaptability, the AFPA technology should provide an essential sensing element to the variety of platforms being developed under the U.S. Army Future Combat System, as well as for the U.S. Air Force and Navy. oe
John Carrano is a program manager at DARPA.
Phil Perconti, Jim Brown
Phil Perconti is division chief and Jim Brown is principal scientist at the U.S. Army Night Vision Laboratory.
Kenneth Barnard is principal scientist at the U.S. Air Force Research Laboratory.