The appearance of any scene varies with lighting direction and viewing direction. Although prints that vary appearance with view direction have been available for several decades now in the form of lenticular prints and holograms, prints that respond to changing lighting direction have only been realized recently.1, 2 For example, a print of a statue can now be made to cast shadows to the left when viewing the print with lighting from the right. When viewing the same print under lighting from the left, the shadows can be made to fall to the right. This interaction is not just limited to shadows. All lighting-dependent effects such as shading, specularity, and subsurface scattering can be simulated and controlled. The added dimension of accurate lighting variability could increase the realism of photographic prints above what is available today.
The appearance of a fixed viewpoint in a scene is given by its reflectance function,3, 4 a 2D function that describes how its intensity varies with lighting direction. To produce a print that responds to lighting as the original 3D scene does, one needs to represent arbitrary reflectance functions at each pixel. Within a scaling factor, each pixel must display the same brightness that the original scene did, depending on the incoming lighting direction.
We developed a simple mechanism that allows this control. Figure 1 shows three pixels in cross section. The configuration consists of a dimpled mirrored substrate, where each dimple corresponds to a single pixel. For each pixel, incoming rays from a light source at a particular direction reach the eye of a viewer reflected off a single point on the substrate. On top of the dimpled substrate, a registered transparency sheet is overlaid, onto which an arbitrary pattern can be printed. If ink on the transparency sheet blocks that reflected ray, the pixel will appear dark for that particular lighting direction. If ink on transparency sheet does not block the ray (see Figure 1), the pixel will appear bright. In this manner, we can modulate which lighting directions produce a return for each pixel. The arrangement shown allows binary control of lighting response. Gray scale can be achieved by applying dithering techniques to the transparency sheet. Unlike conventional printing, dithering can be applied in both lighting space (within a dimple) and spatially (between dimples).
Figure 1. Cross section of three pixels in our lighting-dependent print. Black opaque ink on the surface of a transparency sheet can block reflections corresponding to some lighting directions, but not others (as shown). This allows the specification of which lighting directions make any pixel appear bright or dark.
A fabricated prototype is shown in Figure 2. We constructed the dimpled substrate by forming a geometric inverse of a commercial lenslet array that has 1mm lenses packed hexagonally, effectively using it as a mold. We sputtered aluminum onto this dimpled surface, giving it a mirror-like finish. To make the transparency overlay, we first photographed a real statue, keeping the statue and camera fixed relative to each other, but varying the lighting direction across 50 discrete locations, one for each image. This sampling of the reflectance function for each pixel was then printed onto the transparency sheet with a conventional laser printer, using the dithering methods. Lastly, we carefully aligned the transparency sheet with the dimple array and fixed it in place.
Figure 2. Lighting-dependent prints consist of a dimpled, reflective substrate overlaid with a registered transparency sheet printed with a conventional laser printer.
Although the operation of Figure 1 is discussed in terms of single light sources, due to linearity of lighting across direction, the mechanism described produces the correct effect for the more complex lighting environments that we encounter in everyday life. In addition, the method is extremely tolerant to viewing position, since rotating viewing position roughly approximates rotating the lighting environment, so prints still look realistic.
Photographs of the resulting prototype under two different lighting directions are shown in Figure 3, demonstrating appearance change. The prototype is a simple, low-resolution demonstration of our method, but with sufficient engineering, we believe we can improve the quality to become photorealistic. For instance, full color can be introduced by using triads of reflective red, green, blue pixels. Resolution can be increased from 25 dots per inch (DPI) to 200 DPI using today's commercial printers and still support 16 × 16 reflectance functions. This would enable photorealistic prints that respond to lighting as the photographed objects themselves do, as opposed to conventional prints that display a single lighting condition.
Figure 3. Two photographs of a reflectance print illuminated under different viewing directions.
Palo Alto, CA
Tom Malzbender is a senior research scientist working at the intersections of computer graphics, computer vision, and signal processing. He has developed the techniques of reflectance transformation, polynomial texture mapping, and Fourier volume rendering.
T. Malzbender, R. Samadani, S. Scher, A. Crume, D. Dunn, J. Davis, Printing reflectance functions, ACM Trans. Graphics
31(3), May 2012. Presented at Siggraph 2012
P. Debevec, T. Hawkins, C. Tchou, H. Duiker, W. Sarokin, M. Sagar, Acquiring the reflectance function of a human face, Siggraph
, p. 145-156, 2000.
T. Malzbender, D. Gelb, H. Wolters, Polynomial texture maps, Siggraph
, p. 519-528, 2001.