Ruler enables calibrated image quality measurement
From cell phone cameras to camcorders, digital imaging is ubiquitous in people's lives. Despite the proliferation of consumer options, there are still no standardized objective metrics to quantify the overall performance of these systems. Several International Standards Organization (ISO) standards have quantified different aspects of performance, such as veiling glare,1 opto-electronic conversion functions,2 resolution,3 noise,4 and exposure.5 While these are useful technical specifications, they do not offer a way to combine all of the measurements to predict overall quality.
To overcome this problem, we developed the softcopy ruler method. It enables the calibrated visual assessment of image quality. Because the assessments are measured in just noticeable difference (JND) units, it is possible to combine measurements of different attributes and predict overall quality.6
The softcopy ruler implementation is based on ISO 20462, Part 3.7,8 This work extends the standard by creating reference digital images of known subjective image quality, complementing the hard copy standard reference stimuli.
At the time the standard was written, most displays used cathode-ray tube technology, in which focus drifted over time and modulation transfer function (MTF) varied from the center to the corner of the display. With newer flat panel screens, MTF is consistent over time and spatial position, allowing digital images to be displayed in a well-defined way. This capability enables the production of calibrated, standard images for use in the softcopy ruler.
The method consists of three parts: a set of ruler images developed to meet the requirements set by ISO 20462, a software tool which displays the ruler and test pictures and collects observer responses, and a specified viewing environment. The construction of the ruler images requires quantifying of the MTF of the entire imaging chain, and manipulation of its shape using digital filtering to agree with the reference shape as defined in ISO 20462, Part 3. This process permits us to estimate the absolute quality using the standard quality scale (SQS) defined in the standard.
The SQS scale has one unit equal to one JND of overall quality. For reference, the widths of the categories ‘excellent,’ ‘very good,’ ‘good,’ ‘fair,’ ‘poor,’ and ‘not worth keeping’ average approximately six JND units.9, 10 The zero point of the scale corresponds to an image in which the quality is so low that the principal subject cannot be readily identified. A scale value of 30 falls within the ‘excellent’ region as assigned by consumer photographers. The ruler images we have generated span the SQS range from 2 to 31 in 1 SQS (JND) increments. The set contains a total of 16 scenes depicting varied subject matter (see Figure 1).
Figure 2 shows the graphical user interface. This interface enables users to self-administer a test. It presents a ruler and test image on the left and right-hand side. The perceived sharpness of the sample can be varied by moving the bottom slider bar from left (very sharp) to right (blurred). The test pictures may have different levels of degradation. The software asks the observer to match the test and ruler image in terms of overall quality. For each trial, the program records the position of the ruler image, along with the time used to make the judgment. This ruler position is translated to an SQS value.
The viewing environment includes a high-quality LCD monitor and a headrest to control the viewing distance. In this study we used an Apple Cinema HD Display (30″ flat panel), which has a resolution of 2560 x 1600 pixels and a nominal pixel pitch of 0.25mm. The headrest, made at the University of Houston, College of Optometry, controls the viewing distance. The viewing room was painted neutral gray (see Figure 3). The wall behind the display was uniformly illuminated by D65 fluorescent tubes. Because there is no room light directly hitting the monitor, the displayed image has a high dynamic range. The extended gray field helps to reduce observer fatigue.
We developed the softcopy ruler method during phase 2 of the Camera Phone Image Quality Initiative sponsored by the International Imaging Industry Association (I3A). Since its completion, this method has been used in several studies of individual image attributes. A sample result of a softcopy ruler experiment is shown in Figure 4. In this study, the treatment is on noise reduction, and the image attribute affected is the texture detail. The horizontal axis shows the increase in the strength of the noise reduction filtering. The vertical axis shows the resulting SQS level due to texture blur. Each data point represents an average reading of the slider position across a number of observers and scenes. (A similar technique, called the motion quality ruler method, was developed during the study of digital cinema image quality.11)
The softcopy ruler method provides a new way to assess overall image quality. The I3A will make the pictures, graphical user interface for data collection, and viewing environment specifications available through its website (www.i3a.org). Our next steps will be to revise ISO Standard 20462, Part 3 to reflect these recent advances.
Elaine Jin is the principal imaging scientist at Aptina Imaging. She received a PhD in optical engineering from Zhejiang University in 1989, and a PhD in psychology from the University of Chicago in 1998. Her research interests include vision modeling, human factors in stereoscopic display technologies, and image quality evaluation.
Brian Keelan is chief scientist at Aptina Imaging. He earned his PhD in chemical physics from the California Institute of Technology in 1986. He is the author of the Handbook of Image Quality: Characterization and Prediction, Parts 1 and 3 of ISO 20462 on image quality measurements, and many technical papers. His primary research interests are in computer modeling and image quality.
