Every year, nearly 5,000 pedestrians are killed in traffic incidents in the United States. A system tied into traffic signals that could detect and track pedestrians would be able to decrease this tragic statistic. In addition to traffic safety, pedestrian detection and tracking are important for applications including traffic control, security monitoring, and pedestrian flow analysis. The recent interest in vision-based pedestrian monitoring by the Federal Department of Transportation (DOT) follows the Intelligent Vehicle Initiative (IVI), which has a main goal of preventing the fatalities and injuries that occur in highway crashes.1 An efficient and reliable system of near-real-time outdoor pedestrian detection and tracking is therefore required to serve as input for warning and control systems to be developed within the IVI program.
With funding from the DOT, Migma Systems has developed a new stereo-based vision system to detect pedestrians that fits within the framework of the IVI Program.2 This system is an improvement on previous work that we reported in 2006.3 Our system incorporates significant improvements in a number of areas: a new IR LED stereo camera can detect pedestrians any time of the day or night; a fast disparity-map estimation algorithm provides near-real-time pedestrian detection (2–3 detections per second); a robust image-segmentation method minimizes the impact of camera noise, camera jiggling, and outdoor lighting changes; 3D features are extracted from the disparity map; and a parallel structure offers fast computation.
This pedestrian early-warning system consists of three major components: a stereo camera, a digital signal processor, and a traffic controller. The stereo camera and digital signal processor are packaged as one unit. This system is intended to be deployed at a street intersection, continuously monitoring the crosswalk for people crossing the street. If pedestrians in a crosswalk are detected, it sends a wireless signal to a nearby traffic controller that, in turn, changes the traffic signal phases. As an example, Figure 1 shows Migma's pedestrian detection system deployed at a street intersection.
Figure 1. Migma's pedestrian detection system deployed at a street intersection. (Source: Migma Systems, Inc.)
We built the IR LED stereo camera by arranging two IR LED cameras side-by-side (see Figure 2) such that the focal rays of both lenses are parallel and perpendicular to the stereo baseline, and so that the image planes of both cameras are colinear. This setting ensures that an accurate disparity map can be estimated.
LED emitters illuminate the street crosswalk in the IR, at a wavelength of 850nm. The stereo camera is equipped with 48 IR LED emitters to ensure that the camera has sufficient illumination to provide reliable detection in dim light and at night. A light sensing device automatically turns on or off all the LED emitters, based on the outdoor light intensity. With 48 IR LED emitters, this stereo camera can reliably detect pedestrians 60 feet away, day and night.
Figure 2. The IR LED stereo camera has two lenses with 48 LED emitters around them to provide illumination. (Source: Migma Systems Inc.)
Our objective was to design and develop a pedestrian early-warning system for street intersections that looks for pedestrians 2–3 times each second and operates 24 hours/day, seven days a week. This system is intended to be mounted on top of a traffic light pole 16–24 feet above the ground. It communicates wirelessly with a nearby traffic controller and is designed to operate in all weather conditions. The electronics of the stereo camera will sustain extreme temperatures. We have planned a number of unattended field trials. If the results meet our expectations, the pedestrian detection system will be manufactured and packaged for commercial sale.
Migma Systems, Inc.
Bo Ling is a co-founder and President & CEO of Migma Systems, Inc. He is a Senior IEEE Member (1998) and a part-time faculty member at Department of Electrical and Computer Engineering, Northeastern University, teaching graduate courses of engineering mathematics and digital control. He has given eight presentations at SPIE conferences regarding image and signal processing for both military and commercial applications.