Handheld electronic devices, such as smart phones, have become increasingly popular. However, their LCD backlight consumes a significant amount of energy. While reducing the backlight power prolongs battery life1, it does so at the cost of degrading both luminance and chrominance image quality. Existing image-enhancement algorithms1–6 deal mostly with the luminance appearance, while color has been largely ignored. As a result, we are focusing our work on the chrominance-degradation problem.
We investigated the relation between perceptual attributes and backlight intensity, modeled the effect of backlight using a color appearance model7, and plotted the chroma and saturation versus backlight intensity (see Figure 1). Chroma drops dramatically when we lower the backlight intensity. As a result, the gamut shrinks (see Figure 2).
This observation serves as the basis of our algorithm, which aims to preserve the image's chroma. In the flowchart of our algorithm (see Figure 3), we mixed the two chroma layers using the following equation: where C(x; y) is the resulting chroma; CO(x; y) and CL(x,y), respectively, are the original and backlight-scaled chroma; and wL is a weighting factor. The two problems to be solved are the imaginary color and the display gamut. The former occurs in the inverse International Commission on Illumination Color Appearance Model 02 (CIECAM02) block in Figure 3. The inverse appearance model generates false color in some dark regions: see Figure 4(a). We solved this by setting an upper bound on the variable t in inverse CIECAM02: see Figure 4(b).
The display gamut problem occurs because a visible color is not necessarily displayable. Therefore, we need to map all out-of-gamut colors back into the display gamut. A conventional clipping approach causes a hue shift. A better solution is to project the out-of-gamut pixels to the gamut boundary towards the white point (see Figure 5). Since the constant hue locus is a set of radial curves centered at the white point, our method preserves hue much better.
Figure 1. The variation of chroma and saturation with backlight intensity.
Figure 2. (a) The gamut with full backlight, and (b) with 10% backlight.
Figure 3. The color enhancement algorithm flow chart. CIECAM02: International Commission on Illumination Color Appearance Model 02. sRBG: Standard red-green-blue. LBL: Low backlight.
Figure 4. A demonstration of the imaginary color problem.
Figure 5. A demonstration of the display gamut problem. The magenta curves are constant hue loci. The red points are out-of-gamut. Their gamut-mapped results are in blue.
Figure 6. A demonstration of the algorithm. Left column: 100% backlight. Middle column: 10% backlight, unenhanced. Right column: 10% backlight, enhanced.
Table 1.The Optimal Weighting for Test Images
We carried out subjective tests to find the optimal weight in equation (1). For each of the 20 images in the test set, we generated 11 enhanced images using (1) with different weights. We performed two tests—preference and fidelity. In the preference test, subjects selected the image they liked the most without reference. In the fidelity test, eight subjects were asked to select one of among 11 enhanced images that best matches the full-backlight image. For both tests, we showed images using two HTC Desire smart phones, one with full backlight and the other with 10% backlight.
The averaged weights are shown in Table 1, and some enhanced images are shown in Figure 6. For the unenhanced images, wL equals zero. The higher the wL, the more the chroma is preserved. A negative wL corresponds to a desaturated image. We can see that almost all averaged wL are positive. This indicates that chroma preservation is indeed an effective approach for both faithful and pleasant reproduction.
To conclude, we investigated the effect of low backlight on perceptual attributes and presented a color-enhancement algorithm for images illuminated with extremely low backlight. Our results show that the algorithm effectively compensates for the chroma degradation. Currently, the algorithm involves subjective determination of a weighting factor. In the future, we plan to make the algorithm fully automatic by conducting a thorough analysis of the color perception subjectivity.
Homer Chen, Kuang-Tsu Shih, Tai-Hsiang Huang
Multimedia Processing and Communications Lab
National Taiwan University
1. I. Choi, H. Shim, N. Chang, Low-power color thin-film transistor LCD display for handheld embedded systems, Proc. Int'l Symp. on Low-Power Electron. and Design (ISLPED), pp. 112-117, 2002.
2. F. Gatti, A. Acquaviva, L. Benini, B. Ricco, Low power control techniques for thin-film transistor LCD displays, Proc. Int'l Conf. on Compilers, Architectures and Synthesis of Embedded Syst. (CASES), pp. 218-224, 2002.
3. W. C. Cheng, Y. Hou, M. Pedram, Power minimization in a backlit thin-film transistor LCD display by concurrent brightness and contrast scaling, Proc. Design Automation and Test in Europe (DATE), pp. 10252-10259, 2004.
4. P.-S. Tsai, C.-K. Liang, T.-H. Huang, H.H. Chen, Image enhancement for backlight-scaled thin-film transistor LCD displays, IEEE Trans. on Circuits and Syst. for Video Technol. (TCSVT) 19, no. 4, pp. 574-583, 2009.
5. L. Cheng, S. Mohapatra, M. E. Zarki, N. Dutt, N. Venkatasubramanian, A backlight optimization scheme for video playback on mobile devices, Proc. Consumer Commun. and Networking Conf. (CCNC), pp. 833-837, 2006.
6. T.-H. Huang, K.-T. Shih, S.-L. Yeh, H.H. Chen, Enhancement of backlight-scaled images, 2011. Paper accepted at the August 2011 Multimedia Computing and Commun. (MCC) conference.
7. Mark D. Fairchild, Color Appearance Models, Wiley-IS&T Series in Imaging Sci. and Technol., 2005.