SPIE Membership Get updates from SPIE Newsroom
  • Newsroom Home
  • Astronomy
  • Biomedical Optics & Medical Imaging
  • Defense & Security
  • Electronic Imaging & Signal Processing
  • Illumination & Displays
  • Lasers & Sources
  • Micro/Nano Lithography
  • Nanotechnology
  • Optical Design & Engineering
  • Optoelectronics & Communications
  • Remote Sensing
  • Sensing & Measurement
  • Solar & Alternative Energy
  • Sign up for Newsroom E-Alerts
  • Information for:
SPIE Photonics West 2019 | Register Today

SPIE Defense + Commercial Sensing 2019 | Call for Papers

2019 SPIE Optics + Photonics | Call for Papers



Print PageEmail PageView PDF

Defense & Security

Improved aerial surveillance using onboard image processing

Compact processors installed on small unmanned aerial vehicles could bring the advantages of increased video quality to the battlefield.
31 October 2008, SPIE Newsroom. DOI: 10.1117/2.1200810.1261

Coalition forces in Iraq and Afghanistan use small unmanned aerial vehicles (UAVs) to perform a variety of surveillance and reconnaissance missions.1 To be successful, such missions rely on both the quality of the imagery supplied by the cameras and the ability of image analysts to locate and track objects of interest within the imagery.

Better image quality could provide significant tactical benefits to soldiers on the ground, but if high-resolution cameras were integrated into UAVs, some data would be lost due to communication bandwidth limitations. In addition, higher resolution imagery would take considerably more time to manually analyze because of the denser information content of high-resolution video.

This article identifies our approach to help armed forces conduct their missions more effectively while not requiring higher bandwidth communication links or increasing the image analysis burden.

Current imaging capabilities of small UAVs

The Desert Hawk III2 (DHIII) system is a portable, hand-launchable UAV. It is intended to provide discrete, local aerial video reconnaissance by using its low physical profile and quiet electric propulsion system. Mission profile updates can be made and implemented in flight, giving the DHIII flexibility and adaptability in dynamic situations. Designed principally for covert, real-time video monitoring and recording, it aims to combine versatility, simplicity, and reliability into a highly effective system.

Due to bandwidth limitations of the communications downlink and the necessity for onboard processing, only standard resolution cameras are currently used in the DHIII payload. Any upgrade to higher resolution sensors would require modifications to either the communications system or the image processing system to cope with the increased data flow. Our solution is to preprocess the images onboard the UAV and to transmit the lower-bandwidth data to the user on the ground. This limits the impact on the communications link and does not increase the amount of interaction demanded of the end user.

Modifying a modular payload

The DHIII payload is modular, so individual units can be replaced without affecting other units. An improved system can be created by replacing the standard camera with a higher resolution commercially available unit. The camera is first combined with image preprocessing hardware, which is then installed with the imager in the UAV's modular payload. The system is programmed to implement a library of image-scanning algorithms, selected by the operator on the ground. Executing image preprocessing locally on the UAV enables full analysis on high-resolution imagery without the need to transmit all of the information to the ground.

The higher resolution camera is used in conjunction with a standard analogue version to allow the user greater flexibility to track targets using both region of interest (ROIs) intelligence and visual feedback. The user has the ability to select these ROIs using the ground control station (GCS) software.

The higher quality image is then routed to a processor where ROIs selected using the GCS are detected and annotated onto the video, which is then fed through an image processor to be formatted as a standard National Television System Committee (NTSC) signal. This imagery is transmitted through the payload control board and the video with the annotated ROI is displayed on the GCS monitor.

By keeping all of the processing contained in the payload bay, no modifications are needed on the air vehicle, which provides a seamless upgrade path for current DHIII air vehicles. Figure 1 is a block diagram of the payload system with the addition of a small processing custom field-programmable object array (FPOA) printed circuit board (PCB).

Figure 1. DHIII payload with custom FPOA PCB. EO: electro-optical.
Payload integration

The DHIII turret payload supports two board-level imagers at all times. The imager tray snaps into a pan-and-tilt gimbal which is used to stabilize the video and steer the imagers to the desired location. The turret payload snaps into a DHIII air vehicle for quick deployment as shown in Figure 2. The custom video processing board containing the processing hardware mounts inside the turret payload shell. The rear portion of the payload allows space for a compact PCB to be installed.

Figure 2. DHIII payload.
Algorithm development

The imagery collected by surveillance and reconnaissance systems is time-critical information, so it must be captured and provided with minimum amount of latency. This requires automated image-analysis algorithms.

Selection of ROIs for analysis can be divided into two categories, static and dynamic. Figure 3 shows the approach for static ROI detection based on extracting sub-bands of the wavelet transform.3 These sub-bands are essential in capturing target-related details. Figure 4 shows the approach for dynamic ROI detection. These regions will then be used as inputs to a target tracker.4, 5

Figure 3. Algorithm flow for static ROI detection.

Figure 4. Algorithm for dynamic ROI detection.

Image pre-processing on small UAVs provides an opportunity to simplify target identification. In addition, it reduces the requirements of the communications downlink, which is a size, weight, power, and cost driver on many small platforms. By matching the defined interface control document for the DHIII payload, this feature can be a drop-in upgrade to the current capabilities of the platform.

Rick Stevens, Aaron Cordes, Ryan L. Nelson, Firooz A. Sadjadi, Jacob R. Braegelmann 
Lockheed Martin
Eagan, MN

Rick Stevens is a technical director at Lockheed Martin Tactical Systems and has been involved in the design and development of military processing and interconnect technologies for over 27 years.

Aaron Cordes is a computer engineer for Lockheed Martin Tactical Systems and has been involved in the hardware and software design of advanced processing systems for military platforms.

Ryan L. Nelson earned his undergraduate degree in mechanical engineering at Iowa State University. At Lockheed Martin - MS2 for five years, he has performed mechanical design and analysis for advanced photonics and UAV programs. He has served as lead mechanical designer for the DHIII program for three years.

Firooz Sadjadi is a member of the technical staff at Lockheed Martin Corporation. His interests are in theoretical and experimental research related to signal and image processing, pattern recognition, target tracking, and information fusion.

Jacob Braegelmann is a hardware engineer with Lockheed Martin Tactical Systems. His areas of research include high-performance computing and communication using reconfigurable hardware with an emphasis on solutions for low size, weight, and power.