The work described here fits in the context of a larger project on the objective and relevant characterization of paintings and painting canvas through the analysis of multimodal digital images. We captured, amongst others, X-ray images of different canvas types, characterized by a variety of textures and weave patterns (fine and rougher texture; single thread and multiple threads per weave), including raw canvas as well as canvas processed with different primers. In this paper, we study how to characterize the canvas by extracting global features such as average thread width, average distance between successive threads (i.e. thread density) and the spatial distribution of primers. These features are then used to construct a generic model of the canvas structure. Secondly, we investigate whether we can identify different pieces of canvas coming from the same bolt. This is an important element for dating, authentication and identification of restorations. Both the global characteristics mentioned earlier and some local properties (such as deviations from the average pattern model) are used to compare the "fingerprint" of different pieces of cloth coming from the same or different bolts.

We explored the working methods of the Italian Baroque master Caravaggio through computer graphics reconstruction of his studio, with special focus on his use of lighting and illumination in The calling of St. Matthew. Although he surely took artistic liberties while constructing this and other works and did not strive to provide a "photographic" rendering of the tableau before him, there are nevertheless numerous visual clues to the likely studio conditions and working methods within the painting: the falloff of brightness along the rear wall, the relative brightness of the faces of figures, and the variation in sharpness of cast shadows (i.e., umbrae and penumbrae). We explored two studio lighting hypotheses: that the primary illumination was local (and hence artificial) and that it was distant solar. We find that the visual evidence can be consistent with local (artificial) illumination if Caravaggio painted his figures separately, adjusting the brightness on each to compensate for the falloff in illumination. Alternatively, the evidence is consistent with solar illumination only if the rear wall had particular reflectance properties, as described by a bi-directional reflectance distribution function, BRDF. (Ours is the first research applying computer graphics to the understanding of artists' praxis that models subtle reflectance properties of surfaces through BRDFs, a technique that may find use in studies of other artists.) A somewhat puzzling visual feature-unnoted in the scholarly literature-is the upward-slanting cast shadow in the upper-right corner of the painting. We found this shadow is naturally consistent with a local illuminant passing through a small window perpendicular to the viewer's line of sight, but could also be consistent with solar illumination if the shadow was due to a slanted, overhanging section of a roof outside the artist's studio. Our results place likely conditions upon any hypotheses concerning Caravaggio's working methods and point to new sources of evidence that could be confirmed or disconfirmed by future art historical research.

Considering printed Latin text, the main issues of Optical Character Recognition (OCR) systems are solved. However, for degraded handwritten document images, basic preprocessing steps such as binarization, gain poor results with state-of-the-art methods. In this paper ancient Slavonic manuscripts from the 11th century are investigated. In order to minimize the consequences of false character segmentation, a binarization-free approach based on local descriptors is proposed. Additionally local information allows the recognition of partially visible or washed out characters. The proposed algorithm consists of two steps: character classification and character localization. Initially Scale Invariant Feature Transform (SIFT) features are extracted which are subsequently classified using Support Vector Machines (SVM). Afterwards, the interest points are clustered according to their spatial information. Thereby, characters are localized and finally recognized based on a weighted voting scheme of pre-classified local descriptors. Preliminary results show that the proposed system can handle highly degraded manuscript images with background clutter (e.g. stains, tears) and faded out characters.

Document analysis is done to analyze entire forms (e.g. intelligent form analysis, table detection) or to describe the layout/structure of a document. Also skew detection of scanned documents is performed to support OCR algorithms that are sensitive to skew. In this paper document analysis is applied to snippets of torn documents to calculate features for the reconstruction. Documents can either be destroyed by the intention to make the printed content unavailable (e.g. tax fraud investigation, business crime) or due to time induced degeneration of ancient documents (e.g. bad storage conditions). Current reconstruction methods for manually torn documents deal with the shape, inpainting and texture synthesis techniques. In this paper the possibility of document analysis techniques of snippets to support the matching algorithm by considering additional features are shown. This implies a rotational analysis, a color analysis and a line detection. As a future work it is planned to extend the feature set with the paper type (blank, checked, lined), the type of the writing (handwritten vs. machine printed) and the text layout of a snippet (text size, line spacing). Preliminary results show that these pre-processing steps can be performed reliably on a real dataset consisting of 690 snippets.

Nowadays, 2D photography is the common technique for the documentation and digitalization of historical coin inventories. However, by using 2D photography a huge amount of information is lost due to the projection of a 3D structure onto a 2D image. A solution to this problem would be the use of 3D scanning devices to obtain accurate 3D models of the coins. In this paper we show results of scanning 24 historical coins from the Roman and medieval age using a high-accuracy active stereo scanner. We furthermore highlight the various benefits of this acquisition method for coin documentation, coin measurement and coin recognition. The results show that accurate 3D models can be obtained despite the small size and high reflectance of historical coins.

It has been reported previously that the readability of text in the original Archimedean treatises in the Archimedes Palimpsest is enhanced by spatially normalizing the images and rendering spectral differences in pseudocolor. Though this method worked well for some of the original texts in the palimpsest, the readability of the original text was improved little or not at all on leaves of a commentary on Aristotle's "Categories" by Alexander of Aphrodisias. This is not very surprising, since the various original works within the palimpsest were written by different scribes at various times and places, so the spectral responses of the inks in different manuscripts may be different. However, there is much scholarly interest in the readings from this text, so different image processing techniques were implemented. It was found that significant text information could be recovered from this manuscript by principal component analysis applied to color images of the fluorescence generated under ultraviolet illumination. This result indicates that useful text information is conveyed by the spectrum of the ultraviolet fluorescence. The success of this technique has changed the protocol used in image collections with other manuscripts.

George Eastman House International Museum of Photography Conservation Laboratory and the University of Rochester Department of Computer Science are researching image analysis techniques to distinguish daguerreotype plate and image features from deterioration, contaminant particulates, and optical imaging error occurring in high resolution photomicrography system. The images are captured at up to 30 times magnification and composited, including the ravages of age and reactivity of the highly polished surface that obscures and reduces the readability of the image. The University of Rochester computer scientists have developed and applied novel techniques for the seamless correction of a variety of problems. The final output is threefold: an analysis of regular artifacting resulting from imaging conditions and equipment; a fast automatic identification of problem areas in the original artifact; and an approximate digital restoration. In addition to the discussion of novel classification and restorative methods for digital daguerreotype restoration, this work highlights the effective use of large-scale parallelism for restoration (made available through the University of Rochester Center for Research Computing). This paper will show the application of analytical techniques to the Cincinnati Waterfront Panorama Daguerreotype, with the intent of making the results publically available through high resolution web image navigation tools.

We addressed the problem of finding salient characteristics of artists from two-dimensional (2D) images of historical artifacts. Given a set of 2D images of historical artifacts by known authors, we discovered what salient characteristics made an artist different from others, and then enabled statistical learning about individual and collective authorship. The objective of this effort was to learn what would be unique about the style of each artist, and to provide the quantitative results about salient characteristic. We accomplished this by exploring a large search space of low level image descriptors. The motivation behind our framework was to assist humanists in discovering salient characteristics by automated exploration of the key image descriptors. By employing our framework we had not only saved time of art historians but also provided quantitative measures for incorporating their personal judgments and bridging the semantic gap in image understanding. We applied the framework implementation to the face illustrations in Froissart's Chronicles drawn by two anonymous authors. We reported the salient characteristics to be (HSV, histogram, k-nearest neighbor) among the 55 triples considered with 5-fold validations. These low level characteristics were confirmed by the experts to correspond semantically to the face skin colors.

A number of digital image analysis techniques have been developed in recent years to address art historical questions. These techniques allow non-destructive analyses of art images that can target outstanding problems of attribution, historical ordering, and other stylistic dimensions. However, great care must be taken in designing the comparisons to which these techniques are applied. In this paper, we review recent work by our lab and by others aimed at establishing a toolbox of stylometrics, and we discuss some of the uses and limitations of these methods. We describe a technique that provides robust classification of authentic drawings by Pieter Bruegel the Elder, and we demonstrate new techniques for art historical analysis applied to the works of other masters. Specifically, we demonstrate the use of two low-level statistics (the slope of the log amplitude spectrum and color histogram correlation) to analyze the works of Picasso and Braque. Finally, we show that face detection and recognition techniques may play a useful role in the attribution of works of art. The rationale for employing vision coding-like methods (e.g., sparse coding) in stylometry is also reviewed. We conclude that generic authentication tools are unlikely to provide reliable stylometric predictions but that with careful construction of comparison sets - which we believe must be done in close collaboration with art historians - these techniques provide important predictions that can be weighed against other art historical evidence. We argue further that concurrent predictions derived from analysis of many independent dimensions of image data (e.g., color, luminance, and spatial statistics) provide the strongest evidence for digital stylometric determinations.

The Archimedes Palimpsest imaging team has developed a spectral imaging system and associated processing techniques for general use with palimpsests and other artifacts. It includes an illumination system of light-emitting diodes (LEDs) in 13 narrow bands from the near ultraviolet through the near infrared (▵λ≤ 40nm), blue and infrared LEDs at raking angles, high-resolution monochrome and color sensors, a variety of image collection techniques (including spectral imaging of emitted fluorescence), standard metadata records, and image processing algorithms, including pseudocolor color renderings and principal component analysis (PCA). This paper addresses the development and optimization of these techniques for the study of parchment palimpsests and the adaptation of these techniques to allow flexibility for new technologies and processing capabilities. The system has proven useful for extracting text from several palimpsests, including all original manuscripts in the Archimedes Palimpsest, the undertext in a privately owned 9th-century Syriac palimpsest, and in a survey of selected palimpsested leaves at St. Catherine's Monastery in Egypt. In addition, the system is being used at the U.S. Library of Congress for spectral imaging of historical manuscripts and other documents.

The Library of Congress' Preservation Research and Testing Division has established an advanced preservation studies scientific program for research and analysis of the diverse range of cultural heritage objects in its collection. Using this system, the Library is currently developing specialized integrated research methodologies for extending preservation analytical capacities through non-destructive hyperspectral imaging of cultural objects. The research program has revealed key information to support preservation specialists, scholars and other institutions. The approach requires close and ongoing collaboration between a range of scientific and cultural heritage personnel - imaging and preservation scientists, art historians, curators, conservators and technology analysts. A research project of the Pierre L'Enfant Plan of Washington DC, 1791 had been undertaken to implement and advance the image analysis capabilities of the imaging system. Innovative imaging options and analysis techniques allow greater processing and analysis capacities to establish the imaging technique as the first initial non-invasive analysis and documentation step in all cultural heritage analyses. Mapping spectral responses, organic and inorganic data, topography semi-microscopic imaging, and creating full spectrum images have greatly extended this capacity from a simple image capture technique. Linking hyperspectral data with other non-destructive analyses has further enhanced the research potential of this image analysis technique.

Our paper introduces a system based on digital image processing algorithms designed to facilitate analysis of painting materials during artwork conservation. Microscopic images of minute samples - cross sections - from the artworks are scanned using visible and ultraviolet spectra and under scanning electron microscope. Firstly, the scans are registered to remove geometrical differences. The multimodal nature of the problem led to the application of mutual information. The image quality is maximized by means of blind deconvolution methods. Cross-sections are then segmented to individual layers and distinctive seeds. For the image retrieval part, which facilitates further analyzes and conclusions, the layers are represented by means of wavelet analysis and secondorder statistics. The library of such features can be connected to the time of creation and differences between vectors of the same materials but from different paintings can help during a painter authentication.

We trained generative models and decision tree classifiers with positive and negative examples of the neo-plastic works of Piet Mondrian to infer his compositional principles, to generate "faux" works, and to explore the possibility of computer-based aids in authentication and attribution studies. Unlike previous computer work on this and other artists, we used "earlier state" works-intermediate versions of works created by Mondrian revealed through x-radiography and infra-red reflectography-when training our classifiers. Such intermediate state works provide a great deal of information to a classifier as they differ only slightly from the final works. We used methods from machine learning such as leave-one-out cross validation. Our decision tree classifier had accuracy of roughly 70% in recognizing the genuine works of Mondrian versus computer-generated replicas with similar statistical properties. Our trained classifier reveals implicit compositional principles underlying Mondrian's works, for instance the relative visual "weights" of the four colors (red, yellow, blue and black) he used in his rectangles. We used our trained generative model to generate "faux" Mondrians, which informally possess some of the compositional attributes of genuine works by this artist.

Recent advancements in 3D scanning technology open a window of opportunity in works of art documentation's possibilities. Contrary to classic techniques of visual documentation (a drawing or photograph), 3D scanning may become the first technique offering objective and dispassionate recording of reality because the subjective stage of analysis takes place only during final data processing by end users such as art conservators, historians, archeologists and epigraphers. The general assumption is made that the best representation of digitized work of art is rough measurement data (in many modern cases it is a cloud of points - a set of geometric (x, y, z) data along with additional parameters like color values, surface reflectance etc.). The concept of 3D scanning and data processing has to be designed by an interdisciplinary team, combining technical competency with knowledge of end users' requirements and demands. The basic points of this elaboration are: what additional measurement parameters, beside shape, are needed for full digitization of an object, as well as what accuracy of geometry measurement is high enough for registration of objects made from different materials. This last question is to be answered within a recently started three-year research program, whose methodological assumptions are stated in the presented paper. Some preliminary results are also shown together with discussion of achieved sampling density and accuracy.

We built a full computer graphics model of Parmigianino's studio, including convex mirror, in order to explore the artist's likely working methods during his execution of Self portrait in a convex mirror (1523-4). Our model supports Vasari's record that the radius of curvature of a convex mirror matched the radius of curvature of the wood panel support. We find that the image in the painting is consistent with a simple horizontal rectilinear room drawn from a slightly re-oriented and re-positioned mirror. Our optical analyses lead us to recommend an alteration to the current display arrangement in the Kunsthistorisches Museum.

Common analysis techniques for artworks, such as X-ray based techniques, usually employ high-energy radiation sources. It also oftentimes requires the removal of material from the sample making the analysis relatively destructive. This is unacceptable for samples with high cultural value. Therefore, there is a need to develop alternative nondestructive and noninvasive analysis methods. This paper presents an approach for pigment estimation of Japanese paintings. Reflectance spectra were reconstructed from the RGB values of digital images with the help of multiple linear regression analysis. A reference database with the measured reflectance spectra of the most common pigments used in Japanese artworks was developed and used for identification by comparison and matching. Results have shown that estimation can be successfully performed with only 2% error. The estimation results show some promise that the system could become a powerful tool for the analysis of cultural heritage.

Modern optical measuring systems are able to record objects with high spatial and spectral precision. The acquisition of spatial data is possible with resolutions of a few hundredths of a millimeter using active projection-based camera systems, while spectral data can be obtained using filter-based multispectral camera systems that can capture surface spectral reflectance with high spatial resolution. We present a methodology for combining data from these two discrete optical measuring systems by registering their individual measurements into a common geometrical frame. Furthermore, the potential for its application as a tool for the non-invasive monitoring of paintings and polychromy is evaluated. The integration of time-referenced spatial and spectral datasets is beneficial to record and monitor cultural heritage. This enables the type and extent of surface and colorimetric change to be precisely characterized and quantified over time. Together, these could facilitate the study of deterioration mechanisms or the efficacy of conservation treatments by measuring the rate, type, and amount of change over time. An interdisciplinary team of imaging scientists and art scholars was assembled to undertake a trial program of repeated data acquisitions of several valuable historic surfaces of cultural heritage objects. The preliminary results are presented and discussed.

We present the results of a two-year project aimed at capturing quantifiable color signatures of oil paintings of Fernando Amorsolo, the Philippine's first National Artists. Color signatures are found by comparing CIE xy measurements of skin color in portraits and ground, sky and foliage in landscapes. The results are compared with results of visual examination and art historical data as well as works done by Amorsolo's contemporaries and mentors.

Interests in cultural heritage have grown tremendously in the past few years. These interests vary from preservation, restoration, inspection and archiving just to name a few. Access to cultural heritage is very limited. Therefore, it is important to maximize such opportunity to gather as much information as possible from the cultural heritage. In this study, a technique was proposed to extract analytical information from digitally archived images. The images were acquired at high resolution using a flatbed scanner equipped with a line CCD under fluorescent light illumination. The images were used to reconstruct spectral reflectance using the pseudoinverse method. The results were used for pigment identification and investigation on degradation. Three methods were explored in computing the conversion matrix which contains information from the light source and the camera based on over 600 Japanese pigments as learning samples: (1) use of all pigments in the database; (2) exclusion of some pigments if historical information is available on the sample; and (3) color classification using L*C*H* color space. The technique was applied to the analysis of a real cultural heritage, a hanging scroll painting called Dragon King Zennyo Ryu'o (classified as a Japanese National Treasure, dated 1145) found in Koya Mountain in Japan. The analytical information extracted from the archived images provided insights on the degradation process the painting underwent. In addition, the traces of material detected from the analysis, give art historians scientific proof in creating historical footprints for this precious cultural artifact. This study demonstrated how archived RGB images could be used for the noninvasive and nondestructive investigation of actual cultural heritage.

Statistical analysis of art, particularly of the abstract genre, is becoming an increasingly important tool for understanding the image creation process. We present a multifractal clustering analysis of non-representational images painted by adults and children using a 'pouring' technique. The effective dimensions (D0) are measured for each, as is the associated multifractal depth ▵D = D₀ - D_OO. It is shown that children create paintings whose dimensions D0 are less than those created by adults. The effective dimensions for adult painters tend to cluster around 1.8, while those for children assume typical values of 1.6. In a similar fashion, the multifractal depths for images painted by adults and children show statistically-significant differences in their values. Adult paintings show a relatively shallow depth (▵D ~ 0.02), while children's paintings show a much greater depth (▵D ~ 0.1). Given that the 'pouring' technique reflects the body motions of the artist, the results suggest that the differences in the paintings' fractal characteristics are potential indicators of artist physiology.