New wide-area surveillance techniques for protection of pipeline infrastructure

There are millions of miles of pipes buried along the length and breadth of the United States. The areas through which these pipelines run cannot be used for other activities. Furthermore, machinery—such as construction equipment and heavy vehicles—are major threats to pipeline infrastructure. Monitoring is therefore required to know whether a pipeline's right-of-way is threatened at any time. Rapid advances in sensor technologies have enabled the use of high-end video acquisition systems to monitor the right-of-way of pipelines and have generated a huge amount of data. It is, however, very costly to employ analysts to scan through this data and to identify right-of-way threats. An automated mechanism that can detect these threats and issue warnings is therefore warranted.

Several object detection and recognition algorithms have been proposed previously.^1–5 Car body edges, edges of a front windshield, and shadows can be used as features for car detection.⁶ An alternative simple vehicle detection algorithm involves exploring four elongated edge operators.⁷ A top-down matching method has also been developed for vehicle detection from high-resolution aerial imagery.⁸ On-line boosting, with an interactive training framework, is another option for automatic car detection.⁹ A feature-based approach to car detection uses scale-invariant transform features and an affinity propagation algorithm.¹⁰ We previously designed a three-stage pattern recognition framework to detect construction equipment in various lighting conditions and different object orientations using monogenic signal representation.^{11, 12} The majority of these techniques, however, are either computationally expensive or unable to deal with the complex environments associated with aerial imagery. None of these approaches consider the potential security issues of the detected objects for wide-area surveillance.

We have therefore developed a multistage framework for aerial imagery analysis.¹³ This can be used for the automatic detection and identification of machinery threats to pipeline right-of-way. The three phases involved in our proposed framework can be categorized as background elimination, part-based object detection, and risk assessment. With our approach we can take into account the constraints associated with aerial imagery, such as low resolution, lower frame rate, large variations in illumination, and motion blurs.

We developed the background elimination model with the aim of providing information regarding the contents of an image, which can be used in the process of threat detection. Our proposed segmentation technique has two parts, which are known as local textural features-based segmentation (LTFS) and adaptive perception-based segmentation (APS). LTFS is based on the properties of the area around every pixel within an image. As such, the output of the LTFS only contains prominent information from the input image (e.g., abnormal regions or a fully connected inhomogeneous area). Through LTFS, we are able to eliminate the majority of the background so that only some protruding regions remain (see Figure 1). These regions indicate the possible location of the target. Our LTFS method provides global background elimination, but it cannot be trained to eliminate specific undesired objects (e.g., trees, houses, or roads) that are not considered as threats. We therefore also developed the APS model (see Figure 2). In our APS algorithm, a test image is divided into various segments and passed through the trained model to detect the presence of an object. A sample result of our APS model, in which buildings are being segmented, is shown in Figure 3.

Figure 1. A sample result of the local textural features-based segmentation (LTFS) algorithm. (a) The original red, green, blue (RGB) image. (b) The LTFS output. The yellow circle indicates the threat object that has been identified.

Figure 2. Complete architecture for the proposed adaptive perception-based segmentation (APS) algorithm.

Figure 3. Illustration of an APS algorithm result, in which buildings are segmented.

With aerial imagery, a major challenge for detection is when the object of interest is partially occluded, e.g., by shrubs, trees, or buildings. A part-based model has been demonstrated as an attractive option for object recognition because of its ability to cope with partial occlusions and large appearance variations.^{14, 15} In our proposed part-based model an object is first partitioned into a certain number of parts, depending on the size of the object. We then use local phase information to extract informative attributes for describing the individual parts. Our next step is to convert the object parts into a larger number of clusters. In this process, we group similar parts into the same cluster and a histogram of oriented phase is used to describe the specific pattern of the parts. We subsequently organize each of the detected parts and their attributes as an integrated entity. Since a target can be represented by a certain number of patterns, we can train a classifier to detect the local patterns of the target. An occluded object can thus be detected by parts in the input scene. Multiple parts of an object can be detected without any false alarm with our technique (see Figure 4).

Figure 4. A sample result from the proposed part-based algorithm. The original image is shown in (a) and the part-based detection, with background elimination, is shown in (b). The yellow circle in (a) indicates the location of the object of interest. The red and green regions in (b) are the bigger and smaller parts of the detected object, respectively.

The output of our part-based object detection technique is the pixel location of the threat object in the input image. In real-life applications, however, a pipeline operator must also know the geolocation of an object to prevent any damage to the pipeline. A registration process that links the images to a geographical map is therefore required. In addition, some detected machinery threats may be far from a pipeline, or have a low threat probability (i.e., when they are not being operated). Considering these issues, we have designed an additional framework (see Figure 5) that can automatically analyze the geolocation and temperature information of a detected object, and can assign a risk level to any given threat (high, medium, or low). We have simulated and tested this proposed technique under various sets of challenging conditions to investigate its reliability and feasibility for monitoring the right-of-way of pipelines. In these tests, we have achieved over 85% accuracy for machinery threat detection. We have therefore demonstrated that our method can be used as a practical approach for wide-area surveillance and to protect pipeline infrastructures.

Figure 5. The proposed framework for automatic geolocation, temperature monitoring, and risk assessment of detected objects.

We have designed and tested a novel wide-area surveillance technique that can be used to protect underground pipeline infrastructures. Our multistage framework uses different segmentation and classification approaches to automatically detect machinery threats to pipeline rights-of-way, and to issue warnings regarding the severity of the detected objects. We are currently focusing on hardware-software integration and performance acceleration aspects, so that we can enable real-time processing in an onboard flight environment. We are now planning to modify the detection framework using a cascade of classifiers—with various feature set levels—to eliminate the possibility of false alarms caused by occlusions and cluttered backgrounds, without affecting the detection accuracy.