Three-dimensional modeling of arbitrary surfaces is well established in some niche industrial applications (e.g., animated films and industrial mechanics) and is gaining increasing importance in other fields, such as the documentation of cultural heritage, biotechnology, and medicine. Broad adoption of the technique, however, depends on automating procedures and on being able to obtain reflectance data and artifact-free colors. We describe approaches to satisfying these requirements.
Automated 3D modeling
Constructing 3D models of generic surfaces involves the so-called 3D modeling pipeline,1 which encompasses the four steps shown in Figure 1. The most critical and challenging of these is coarse registration: that is, finding correspondences in two partially overlapping views. These are then used to compute the rigid movement (rototranslation) aligning them. Coarse registration is the only pipeline task that is not solved automatically. Indeed, it typically requires human supervision or the aid of markers.
Figure 1. The 3D modeling pipeline.
We propose instead a fully automated approach to 3D registration, and consequently to the entire modeling procedure. This means automating the coarse estimate of the overlap between a pair of 3D views. Our solution is based on an extension of the ’spin-image’ (SI) method,2 which includes color and texture. An SI is basically an image of the shape of an object seen from a reference system centered on a point on the object's surface. A 3D view makes it possible to build the SIs of a subset of the object's points and to compare them with a set of SIs of a partially overlapping view. The object will look similar seen from nearby positions. Thus, matching points in the two views generate similar SIs, and one can estimate the overlap.
Unfortunately, SIs (or similar shape descriptors) inevitably fail to align views from highly symmetrical (e.g., spherical or cylindrical) or flat objects (e.g., walls). For this reason we propose a new descriptor, the ‘textured spin image’ (TSI), which takes into account not only the shape of the object but also its texture.3 TSIs are therefore able to solve the problem of automatic 3D view registration for flat frescoed walls and for objects of cultural interest with highly symmetrical shapes, such as ancient vases or situlas.
Color in 3D modeling
Typical texture artifacts caused by vignetting—darkening of the edges of an image due to the optics of the camera lens—and different illumination conditions between subsequent acquisitions can already be addressed using the methods of Andreetto and colleagues3 (see Figure 2) or more effectively by the wavelet-based method derived from Zanuttigh's team.4 Though such approaches are adequate for the artifact-free rendering of color appearance, accurate color measurements require multispectral information, that is, the acquisition of an appropriate number of samples of the spectral reflectance of an object.5 For 3D models this requires suitable instruments, such as that described by Brusco and co-workers.6 This couples a range finder with a spectrograph operating in the visible and near-infrared spectra. These techniques enable faithful shape and color reproduction, as shown in Figure 3. Further, 3D modeling systems based on geometry and multispectral data can render valuable service by repeatedly measuring the conservation status (e.g., oxidation, pollution, geometry) of frescoed buildings over time, thus facilitating monitoring for restoration planning.
Figure 2. Examples of (a and b) overlapping textures and (c) blended texture.
Figure 3. A view of the 3D model of the Scoletta del Carmine in Padova, Italy.
In addition to routine 3D modeling issues, multispectral analysis requires the solution of a number of nonstandard tasks, such as controlling the position and spectral content of illumination sources, planning the setup of the acquisition system, and manipulating and analyzing multispectral data. Our current efforts aim to create simple 3D modeling procedures capable of handling geometry and multispectral data.
Automated 3D modeling methods are important because they reduce the time an operator must devote to the modeling procedure, enable nontechnical personnel to perform 3D modeling, and allow repeated measurements. Color features are a major issue in cultural heritage. Multispectral color, in connection with 3D geometry, is a new approach that requires appropriate acquisition instruments and modeling tools. State-of-the-art solutions can certainly be improved with respect to speed and ease of use. Our current efforts are focused both in this direction, and on the parallel fields of information analysis and standardizing acquisition procedures.
Anna Paviotti, Nicola Brusco, Guido Maria Cortelazzo
Department of Information Engineering
University of Padova
Anna Paviotti was born in Udine, Italy, in 1980. She graduated in telecommunications engineering from the University of Padova in October 2004. Since January 2005 she has been a PhD student at the Department of Information Engineering of the same university.
Nicola Brusco was born in Caprino Veronese, Italy, in 1977. He graduated from the University of Padova with a degree in computer engineering in July 2002. In December 2005 he obtained a PhD in telecommunication engineering at the Department of Information Engineering of the University of Padova, where he is currently holding a postdoctoral fellowship.
Guido M. Cortelazzo was born in Este, Italy, in 1952. He holds a degree in electrical engineering from the University of Padova (1976), and MS and PhD degrees in electrical engineering from the University of Illinois at Urbana-Champaign (1980 and 1984, respectively). From 1983 to 1986 he worked with M/A-COM Linkabit, a satellite communication company based in San Diego, CA. In 1986 he joined the Department of Information Engineering of the University of Padova, where he is now full professor.
6. N. Brusco, S. Capeleto, M. Fedel, A. Paviotti, L. Poletto, G. M. Cortelazzo, G. Tondello, A system for 3D modeling frescoed historical buildings with multispectral texture information, Mach. Vis. Appl. 17, pp. 373-393, Springer, 2006.