Detecting muzzle flashes with multispectral imaging may increase soldier survivability

Soldier survivability on the battlefield requires engaging threats before the situation turns lethal. But before threats can be engaged, they must be detected. Detecting a muzzle flash or weapon firing is one way to achieve hostile fire indication (HFI). HFI systems that detect small arms or rocket-propelled grenades, when combined with an adequate countermeasure, increase the chances that personnel and equipment will survive an enemy encounter. However, the HFI systems currently in development are generally limited by cost, weight or size, and/or field of view.¹

Multispectral systems have been proposed for HFI because the radiant intensity of the bright flash at the muzzle of fired weapons has been shown to be wavelength dependent. However, multispectral system components are typically expensive, heavy, and power intensive. In addition, some multispectral sensor materials force the cameras to be kept at cryogenic temperatures,^1,2 which limits their use in the field. However, silicon-based sensors can operate at ambient temperatures, and recent advances have increased their speed and decreased their cost,^3,4 power, size, and weight, thus making them an attractive candidate for a multispectral-based muzzle flash detection system.

Figure 1. Relative counts of FluxData's dual-bandwidth multispectral imaging system with dotted lines showing 90^°field-of-view limits.

During the past four years, we have developed a test bed and procedure for assessing whether multispectral imaging systems can be used for HFI.⁵ This work is a collaboration between the Army Research Laboratory's Survivability/Lethality Analysis Directorate (SLAD), which conducts analytical investigations to ensure that U.S. personnel and equipment survive and function effectively in hostile circumstances, and FluxData Inc., a company that develops multispectral cameras. We have determined the key limitations and components needed for muzzle flash detection and have used multispectral hardware developed by FluxData to produce experimental data for analysis.

Using our HFI test bed, we can examine a multispectral imaging system and assess its ability to accurately detect muzzle flashes with a low false-alarm rate. Within a dual-band multispectral imaging system, light from the source follows two distinct paths to the image plane and then through the image processing pipeline. It is critical that the two bands and their corresponding pixels follow the same processes to ensure accurate image analysis and comparison. Therefore, we first determine the location of each corresponding pixel in both bands, a process called spatial registration, so that we know where the energy from each band is located. Next, we examine the size of the system's field-of-view and confirm that the instantaneous field-of-view is sufficient for muzzle flash detection. This includes ensuring that there is sufficient signal intensity for muzzle flash detection at the operational range and accounts for any effects from atmospheric attenuation. Next, we test for any spectral shift from the optical process that is not part of the dual-band architecture. For example, we verify that as we move to the edge of the field-of-view, the spectral response of the system stays within operational limits. In addition, we verify that the system exhibits a sufficient frame rate and synchronized alignment of the bands in time. Finally, we examine the system's size, weight, power, and cost (SWaP-C). We assume that if all of the above requirements are met (except for SWaP-C), then high-accuracy, low false-alarm rate muzzle flash detection is possible with the system.

Figure 2. FluxData dual-band multispectral imaging system sensor head (left) and processor (right). The wide field-of-view dual-band multispectral system addresses a previous equipment shortfall for multispectral imaging systems, an operational field-of-view of at least 90^°.

One system we investigated using our HFI test bed was FluxData's two-megapixel multispectral imaging system. The system consists of a short focal-length lens that produces, captures, and processes two real-time images at different wavelengths, while exhibiting a minimal peak sensitivity shift across a 90^° field-of-view (see Figure 1). Light from a captured scene first passes through the lens and then through an optical system that includes a novel beam splitter and dual-bandpass spectral filter. The optical system forms two images of the scene side-by-side on a single CMOS sensor. The design of the optical system ensures that the two images are spatially aligned so that pixel-wise comparisons of the left and right images can be used to analyze differences in the two spectral bands. Because the image pairs are captured on a single sensor with a global electronic shutter, they are also synchronized in time. Spectral and temporal changes in the scene can be compared with the known properties of muzzle flashes and used to detect and locate weapons fired within the field-of-view of the system.⁶

The resolution of the system may not seem significant when digital single-lens reflex cameras often exhibit resolutions 10-times higher and cell phone cameras possess two or three times higher resolution. However, this dual-band wide field-of-view imaging system runs at 340 images per second, producing a data stream of 680MB/s. Due to advancements in sensor designs, central processing units, and solid state storage drives, the computer that provides real-time muzzle flash detection and archival storage of the raw data for further research and analysis weighs only a few pounds and is 8.5in×11in×4.5in in size (see Figure 2).

The wide field-of-view dual-band multispectral system designed by FluxData addresses an equipment shortfall for dual-band multispectral imaging systems, which typically lack an operational field-of-view of at least 90^°. Its size, weight, and power are also reduced compared to what was previously available. As the frame-rate capabilities of these multispectral imaging systems increase, faster transient events will be included in the analysis.

The role of the Survivability/Lethality Analysis Directorate is to provide solutions for survivability of soldiers and equipment using analytical technical processes, and to ensure that data is available from these technical processes for analysis. The test bed described in this article provides a tool that analyzes data from the imaging systems being tested to determine their ability to detect muzzle flashes. Future work will include updating our technical processes and test bed using the lessons learned.