Improved aerial surveillance using onboard image processing

Coalition forces in Iraq and Afghanistan use small unmanned aerial vehicles (UAVs) to perform a variety of surveillance and reconnaissance missions.¹ To be successful, such missions rely on both the quality of the imagery supplied by the cameras and the ability of image analysts to locate and track objects of interest within the imagery.

Better image quality could provide significant tactical benefits to soldiers on the ground, but if high-resolution cameras were integrated into UAVs, some data would be lost due to communication bandwidth limitations. In addition, higher resolution imagery would take considerably more time to manually analyze because of the denser information content of high-resolution video.

This article identifies our approach to help armed forces conduct their missions more effectively while not requiring higher bandwidth communication links or increasing the image analysis burden.

The Desert Hawk III² (DHIII) system is a portable, hand-launchable UAV. It is intended to provide discrete, local aerial video reconnaissance by using its low physical profile and quiet electric propulsion system. Mission profile updates can be made and implemented in flight, giving the DHIII flexibility and adaptability in dynamic situations. Designed principally for covert, real-time video monitoring and recording, it aims to combine versatility, simplicity, and reliability into a highly effective system.

Due to bandwidth limitations of the communications downlink and the necessity for onboard processing, only standard resolution cameras are currently used in the DHIII payload. Any upgrade to higher resolution sensors would require modifications to either the communications system or the image processing system to cope with the increased data flow. Our solution is to preprocess the images onboard the UAV and to transmit the lower-bandwidth data to the user on the ground. This limits the impact on the communications link and does not increase the amount of interaction demanded of the end user.

The DHIII payload is modular, so individual units can be replaced without affecting other units. An improved system can be created by replacing the standard camera with a higher resolution commercially available unit. The camera is first combined with image preprocessing hardware, which is then installed with the imager in the UAV's modular payload. The system is programmed to implement a library of image-scanning algorithms, selected by the operator on the ground. Executing image preprocessing locally on the UAV enables full analysis on high-resolution imagery without the need to transmit all of the information to the ground.

The higher resolution camera is used in conjunction with a standard analogue version to allow the user greater flexibility to track targets using both region of interest (ROIs) intelligence and visual feedback. The user has the ability to select these ROIs using the ground control station (GCS) software.

The higher quality image is then routed to a processor where ROIs selected using the GCS are detected and annotated onto the video, which is then fed through an image processor to be formatted as a standard National Television System Committee (NTSC) signal. This imagery is transmitted through the payload control board and the video with the annotated ROI is displayed on the GCS monitor.

By keeping all of the processing contained in the payload bay, no modifications are needed on the air vehicle, which provides a seamless upgrade path for current DHIII air vehicles. Figure 1 is a block diagram of the payload system with the addition of a small processing custom field-programmable object array (FPOA) printed circuit board (PCB).

Figure 1. DHIII payload with custom FPOA PCB. EO: electro-optical.

The DHIII turret payload supports two board-level imagers at all times. The imager tray snaps into a pan-and-tilt gimbal which is used to stabilize the video and steer the imagers to the desired location. The turret payload snaps into a DHIII air vehicle for quick deployment as shown in Figure 2. The custom video processing board containing the processing hardware mounts inside the turret payload shell. The rear portion of the payload allows space for a compact PCB to be installed.

Figure 2. DHIII payload.

The imagery collected by surveillance and reconnaissance systems is time-critical information, so it must be captured and provided with minimum amount of latency. This requires automated image-analysis algorithms.

Selection of ROIs for analysis can be divided into two categories, static and dynamic. Figure 3 shows the approach for static ROI detection based on extracting sub-bands of the wavelet transform.³ These sub-bands are essential in capturing target-related details. Figure 4 shows the approach for dynamic ROI detection. These regions will then be used as inputs to a target tracker.^{4, 5}

Figure 3. Algorithm flow for static ROI detection.

Figure 4. Algorithm for dynamic ROI detection.

Image pre-processing on small UAVs provides an opportunity to simplify target identification. In addition, it reduces the requirements of the communications downlink, which is a size, weight, power, and cost driver on many small platforms. By matching the defined interface control document for the DHIII payload, this feature can be a drop-in upgrade to the current capabilities of the platform.