Recognize 3D objects with passively acquired imagery

Both military and industrial applications can use automatic object recognition, which seeks and identifies specific objects. The task is complicated by distortion and noise captured by the sensors, but techniques have been developed to improve the recognition performance. Many of these use 2D object recognition, but there is growing interest in developing a 3D system.

If one wants to automate object recognition, then the object's 3D outline is very useful. The 3D appearance of an object is usually very stable: it does not depend on the time of day, position of the sun, season, or weather. Therefore, using 3D shape overcomes some of the problems of intensity-based 2D passive electro-optical (EO) or IR imaging.

However, this approach doesn't solve every shape-finding problem because the target's 3D shape can be changed. For example, brush can be piled on top of a tank. Nevertheless, 3D shape recognition provides a number of advantages compared with traditional 2D imagery, simply because we know the 3D position of every point in the image. This allows one to separate the object of interest from the background, obscurants, and clutter. In a 2D image, these would all contribute signal to the same sensor pixel and one would not be able to differentiate among them.

Another significant benefit of the 3D approach is the enhanced performance of target-recognition algorithms. Not only can the algorithms operate on the basis of images containing less background and clutter, the additional dimension also provides an increase in the pixel count, thus producing significantly better performance (such as sharper correlation peaks). Finally, the 3D shape can easily be retrieved using any sensor that measures two of the three dimensions, while a 2D shape is sometimes difficult to correlate because the two dimensions measured are not always the same ones.

Figure 1. Using the passive 3D image acquisition system at the top, a 3D reconstruction detects targets (the cars) despite occluding objects (the tree). The lens array in front of the camera creates multiple images of the scene, each from a known and unique location. Computational reconstruction of the 3D image depends on the differences in perspective between each 'elemental image.' (Image courtesy of B. Javidi.)

We traditionally use laser radar for 3D shape detection. The distance to an object is estimated by measuring the time it takes for the reflected energy to return to the receiver. Angle-angle (for example, azimuth-elevation) or cross-range target dimensions are obtained either through laser-beam scanning (if the receiver consists of only a single or a limited number of detectors) or by large-area flood illumination and subsequent detection of the scattered return signal on a 2D focal-plane array. This technique uses controlled illumination, which means that the sensor can operate at times and in locations that may be beyond the capabilities of other EO sensing modes. In addition, one can choose to receive data only when the scattered return from the object of interest is expected to arrive. This approach neglects the backscatter from intervening obscurants, and hence it results in a higher-contrast image. If this type of 3D imaging at short range can be done without a laser, why use one? Eliminating the laser would reduce the system's cost, weight, size, and power requirements.

We explored¹ an approach to estimate the 3D structure of an object derived from passively acquired imagery. ^2,3,4,5 Figure 1 illustrates the steps involved in the underlying concept. A lenselet array in front of the sensor produces multiple images of the scene, each from a slightly different position. The concept is similar to stereoscopic cameras that record two images: when we view the stereo images, our brains use the difference between the two perspectives to reconstruct distance cues. When we provide a computer with many 'elemental images' each with its own unique perspective, optical or numerical processing can reconstruct the 3D scene. Because this method depends on cross-range resolution, it provides a very acceptable volume image at short ranges.

We compared the image quality of our work to both scanned and flash-laser-radar techniques. For short-range (and especially low-cost) applications, the passive approach to 3D imaging is very desirable. Passive 3D images may provide small voxel (in other words, cubic-pixel) volumes at short range and therefore high detection probabilities, without the expense or weight of a laser illuminator. The passive approach can 'see around' occlusions in a manner similar to 3D active imaging. We will continue to develop passive 3D imaging as an alternative to the equivalent active techniques for short-range and inexpensive applications.