Human motion monitoring in sensitive environments

Closed circuit cameras are a common feature in everyday life. But in some environments, such as homes and prisons, human movement may be best monitored without high resolution cameras. Multi-spectral sensors can be used to view objects of interest at low resolution, avoiding unnecessary invasion of privacy. By not resolving details, the perception of intrusion can be minmized but activity still monitored.

The conventional method for detecting moving objects by spatial image processing is by analyzing the differences between successive or neighboring image frames. But such an option is not viable in situations where low resolution provides insufficient spatial detail. By adapting techniques from the field of multi-spectral remote sensing, however, we have devised an alternative method that achieves detection with a small number of pixels and uses low-resolution sensors.

For our purposes, ‘color’ is a general term signifying intensities of multiple light wavebands in satellite imagery—that is, in the visible and infrared regions of the electromagnetic spectrum. For a simple color image sensor, three channel intensities can be obtained from the red-green-blue values. Using spectral unmixing techniques, the color information for the low resolution sensor array can be used to decompose the data into separate contributions. For operation in indoor environments and to contend with the effects of rapid changes in illumination (such as switching lights on or off), robust algorithms are crucial

Figure 1. Center of object (person in a green shirt) is shown by white cross (frame 1, left) obtained after pixel decomposition. Frame 90 (right) shows trail of center-of-object positions tracking movement.

Spectral unmixing is based on a linear mixing model (LMM) that takes each pixel in a given image as containing a proportion of one or more pure colors known as ‘endmembers’.¹ Thus, each mixed pixel may be decomposed into a linear combination of the individual colors. The difficulty lies in automatically finding and tracking endmembers. This can be achieved by fitting a simplex around the data and identifying its vertices. The relative distance between the vertices and each data point gives an indication of the relative contribution of that color to the pixel value.

Practical application of this model required an automated technique for subjecting each frame in a sequence of images to simplex wrapping. We have developed an optimization procedure that can contend with situations featuring both slow continuous and rapid changes in the color and intensity of ambient lighting. The objective was twofold: to fit a simplex around the greatest possible number of data points (ignoring noisy outliers) while keeping the simplex closely wrapped around the data points. Our solution uses a genetic algorithm to perform this continuous minimization process with a population of vertices that propagates from frame to frame. As lighting conditions change, the population of vertices quickly adapts to respond to the changes.²

Figure 2. Simplex wrapping and un-mixing for three color bands in one frame of data.

To evaluate the algorithms, we performed a number of experiments with human subjects. We recorded different paths of people moving in an enclosed space, developed diverse scenarios with multiple occupants, changed lighting conditions, and varied levels of color contrast between foreground and background. An example using a 30×30 sensor array (see Figure 1) shows the continuous track generated by a person moving around the environment. Figure 2 shows the endmember unmixing, or process of decomposition. In all scenarios the adaptive spectral unmixing algorithms proved reliable and robust.

This demonstration of remote sensing techniques in indoor environments with highly dynamic lighting conditions represents a significant technological development that will underpin non-intrusive sensor development programs at the University of Liverpool. Algorithms that further these aims are expected to contribute to the monitoring systems developed at the Centre for Intelligent Monitoring Systems (CIMS).³ In domiciliary care environments, research at CIMS has already fielded several low-resolution sensor arrays. These systems have operated for months at a time and should enable us to derive behavior models that can further improve the detection and monitoring performance of the algorithms.