Computer algorithms that mimic human vision must respond to the spatial content in images

A growing number of spatial color algorithms can achieve color constancy and dynamic range compression.
13 May 2007
John McCann and Alessandro Rizzi

Human vision has remarkable image-processing power. For instance, it captures information over a very wide dynamic range of light intensities and spectral distributions. Unlike films and electronic sensors, visual appearances are nearly constant, despite widely variable input stimuli. Computer algorithms mimic vision by responding to the image content, as well as to the radiometric properties of individual pixels. The spatial analysis of images is the basis of appearance constancy, with both changes in spectral content and the level of light.

Many techniques are used to render the quanta from a scene recorded at each pixel in an image. The first is based on the experiments of Ferdinand Hurter and Vero C. Driffield in 1890.1 They measured the photographic density in images as a function of the light at a pixel. Such plots are now known as H&D curves. The simplest tool in portraying scenes is to optimize the shape of the H&D curve, also called a tone scale function. These functions are considered global because they have the same effect on all pixels with the same quanta catch.

In 1968, E. H. Land, working with scenes under nonuniform illumination, showed that a white paper in the shade had the same luminance as a black paper in direct light.2 The range of pixel luminances was 1000:1, that is, 33 times more than the range of reflectances in a print. Adjustments of the H&T tone scale are futile in such cases, because H&D curves that improve the rendition of the white paper by making it lighter worsen the black. Conversely, curves that improve the rendition of the black paper by making it darker worsen the white. Global functions cannot improve both the white paper in the shade and the black paper in direct light. Land observed that these two papers with the same luminance appeared white and black. His model for human vision built sensations by combining local spatial comparisons across the entire image. His model, called retinex, was responsive to local changes in luminance, but independent of luminance at a given pixel.

Nowadays, there is a growing family of algorithms that can treat, modify, or enhance color information in its visual context. They are known as spatial color methods, e.g., retinex,3 ACE (automatic color equalization),4 or RSR (random spray retinex).5 These models are responsive to image content as well as to pixel statistics. They produce results that, due to a changing spatial configuration, can have a non-unique relationship with the physical input. For this reason, they cannot be described using convolution filters. And since their behavior changes according to the image content, their impulsive response is not fixed.

They are all based on recomputing the color of each pixel through the spatial distribution of values in the image, but many differences arise depending on their purpose.6 In this perspective, spatial color algorithms (SCAs) can be classified in three main groups: those that accurately model the human vision system (HVS) and predict color appearance (SCA-HVS models); those aimed at enhancing images in the direction of human visual appearance (SCA rendering); and those that attempt to calculate the actual reflectance of an object from the radiance, i.e., the reflectance*illumination (SCA reflectance). Since SCAs can have three distinct goals, three different types of outcomes are expected, and three different types of performance measurements are required.

There are a number of important examples in which human appearance does not correlate with reflectance. The simplest example is simultaneous contrast: humans, SCA-HVS models, and SCA rendering all report different grays on white and black surrounds, while SCA reflectance reports them as identical. SCA-rendering algorithms compute only a qualitative estimate of the final visual appearance, while SCA-HVS models aim also to predict it quantitatively. Quantitative measurements of how accurately SCA models are able to predict the HVS visual response is the metric for these algorithms.

Judging the performance of these models is a challenging task and remains an open problem. Two main variables affect the final result of these algorithms: their parameters and the visual characteristics of the image they process. The term ‘visual characteristics’ refers not only to the digital pixel values of the image (e.g., calibration of pixel value, measured dynamic range of the scene, measured dynamic range of the digital image), but also to their spatial distribution in the image. We recently discussed the visual configurations in which spatial color methods show interesting or critical behavior.6 We surveyed the more significant configurations, including color constancy and contrast. We also described the respective strengths and weaknesses of the different algorithms to provide a deeper understanding of their behavior in the hope of stimulating discussion on the search for a common evaluation ground.

John McCann
McCann Imaging
Belmont, MA 

John McCann received a BA in biology from Harvard University in 1964. He managed the Vision Research Laboratory at Polaroid from 1961 to 1996. He has studied human color vision, digital image processing, large-format instant photography, and the reproduction of fine art. He currently does consulting and research on color vision. He is a fellow and past president of the Society for Imaging Science and Technology, was an honorary member in 2005, and a recipient in 2002 of its Edwin Land medal.

Alessandro Rizzi
Dipartimento di Tecnologie dell'Informazione,
Università degli Studi di Milano
Crema, Italy

Alessandro Rizzi obtained a degree in computer science at the University of Milan and received a PhD in information engineering at the University of Brescia, Italy. He is now an assistant professor at the University of Milan, teaching courses on multimedia and human-computer interaction. Since 1990, he has been carrying out research in the field of digital imaging and vision. His main research topic is the use of color information in digital images, with particular focus on mechanisms of color perception.

Recent News
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research