Reducing the false alarm rate in abandoned object detection

Rapid detection of unattended packages left in public places is important to prevent security threats to people and property. Unfortunately, constant surveillance of public domains is challenging and labor-intensive. Therefore, improved automated abandoned object detection systems are increasingly in demand. An ongoing challenge in computer-aided detection is interference of benign stationary objects—such as people sitting on benches—being flagged as suspicious. The reason for this is that ‘abandoned items’ are usually defined as objects newly added to the scene that have remained stationary for a predefined length of time. As a result, any immobile object can be flagged as suspicious, causing a high number of false alarms. These costly incidents could be prevented by classifying suspicious objects as either living or non-living. Here, we fuse images from thermal and optical cameras to filter out living objects and reduce the false alarm rate.¹

Our system for abandoned object detection was designed to assist operators surveilling indoor environments, such as airports, railway, or metro stations. We use the information obtained from thermal and optical cameras observing the same area for this process. The images from each camera are processed separately to detect abandoned objects. We apply an additional step to the images from the thermal camera to discriminate animate from inanimate objects. Then, the abandoned object detection results are fused: if the abandoned object is not radiating heat (i.e., is classed as non-living), the alarm is raised.

Figure 1. Block diagram of our proposed abandoned object detection system. Thermal images undergo an additional processing step to detect living subjects in the scene.

Figure 1 shows the main steps of our proposed method. We apply background subtraction,² abandoned object detection,³ and post-processing to both thermal and optical image data sets to extract the abandoned objects from the scene. We extract living objects by processing the thermal images. Since living subjects radiate heat, they are expected to look brighter in the thermal images—in a typical white-hot setting—than inanimate objects. To improve the discrimination of hot objects, we use local intensity operation (LIO), which brightens bright pixels and darkens dark pixels.⁴ Finally, we use the abandoned object results in combination with the living subject results to confirm the presence of stationary people in the scene, eliminating false alarms as a result.

Figure 2. Screen captures from a scene with a heating radiator and unattended luggage on a high-reflectance floor. The luggage is flagged as suspicious and non-living (green), and the stationary person is marked as living (red). As the radiator is captured by both cameras, it is not marked as suspicious.

We tested our method using a contrived scene with a hot radiator in the field of view of the cameras (see Figure 2). In the sequence, we detected luggage (shown in green) as a suspicious abandoned object and a person as living object (shown in red). The radiator was segmented as a hot object in the LIO operation. However, because it was a part of the background in both the visible and thermal domains, it was not detected as suspicious. That is, our method did not produce any false alarms from hot objects in the environment.

Not only were we able to discriminate abandoned objects from people, but we also increased the true detection rate of inanimate objects. For example, items left in shadow might not be detected in the visible band because of low contrast against the background. However, if the thermal contrast is sufficient, these items could potentially be detected in the thermal band. Additionally, we compensate for surface reflection in using the two cameras. For example, a surface, such as the floor, can reflect thermal radiation. As a result, the reflection appears in the thermal image. Since we use an optical camera as well, we can eliminate this problem.

In summary, we have developed a method for extracting living objects from a surveillance scene using thermal imaging. This reduces the number of false alarms raised from sedentary objects and improves the rate of true detection of suspicious inanimate objects. In future work, we aim to run our algorithms on graphics processing units, which will significantly reduce the number of computers required for these operations. We are also currently extending the system by adding a capability that will allow the tracking of living and non-living objects separately. This will allow us to detect and follow a person who left a suspicious object.