Optics for Arts, Architecture, and Archaeology (O3A) IX

We discuss applications of pump-probe microscopy to the investigation of pigments in artworks. In pump-probe, a series of ultrafast laser pulses probe molecular state dynamics, which is governed by several physical mechanisms that evolve on timescales ranging from sub-picoseconds to many microseconds. The interplay of these mechanisms can lead to complex dynamics and is sensitive to a pigment’s structure, composition, and environment. We describe strategies to interpret pump-probe dynamics and to disentangle the underlying contributions from different physical mechanisms. We also report on applications to the studies of Cadmium pigments, carbon-based black pigments, and their degradation mechanism.

Varnishes made of natural terpenoid resins are vulnerable components of paintings since they can undergo complex and differentiated chemical and structural changes over time depending on the type of varnish and the conservation conditions. The present work aims at studying the in-depth effects of natural and artificial aging on terpenoid varnish layers using nonlinear optical microscopy (NLOM) in its modality of multiphoton excitation fluorescence (MPEF) complemented with laser-induced fluorescence (LIF) and optical microscopy measurements. To this end, solvent-based terpenoid varnishes with different thicknesses and degrees of aging were considered. A homemade nonlinear optical microscope, based on a tightly focused pulsed femtosecond laser emitting at 800 nm, was used for the MPEF investigation. Single-photon LIF measurements served to determine the degree of surface aging and the optimum NLOM-MPEF operating conditions and helped to interpret the results obtained by applying the latter. The LIF results showed an increase of fluorescence intensity and a red shift of the maximum of emission for varnishes subjected to longer aging periods and of thinner layers. The purely non-invasive NLOM-MPEF approach serves well to assess the in-depth-dependent degradation gradients across the thickness of the layers and its dependence on the aging time and the thickness of the varnish layer. Acknowledgments This research has been funded by the Spanish State Research Agency (AEI) through project PID2019-104124RB-I00/AEI/1013039/501100011033, by the H2020 European project IPERION HS (Integrated Platform for the European Research Infrastructure ON Heritage Science, GA 871034), and by the Community of Madrid project Top Heritage-CM (Tecnologías en Ciencias del Patrimonio, S2018/NMT_4372). Support by CSIC Interdisciplinary Platform “Open Heritage: Research and Society” (PTI-PAIS) is acknowledged.

Analytical techniques such as macro X-ray fluorescence can capture dense multi-dimensional data that provides unique quantitative information at each point over the surface of a work of art. The processed outputs from scanning Macro-XRF are the elemental distribution maps that provide measurements of the abundance of the constituent elements at each point. The ability to visualize and correlate this data with other imaging modalities is an important step in understanding the material composition of a work of art. However, this data is often in a form which is hard to use directly or integrate with the results from other complementary analytical techniques. For example with the multi-dimensional data produced by reflectance imaging spectroscopy, or even with other standard scientific imaging modalities. As a result, Macro-XRF data is often handled and processed separately and the results only made accessible in the form of exported image renderings. In this paper, therefore, we will examine how Macro-XRF data can be processed into a practical and re-usable form that is compatible with other imaging modalities and how this data can be made more easily available through an integrated platform for multi-modal imaging. Open source software will be presented which implements this architecture, showing how quantitative Macro-XRF data can be visualized interactively through a web browser, integrated with other imaging modalities and how the underlying quantitative data can be made accessible and re-used using open formats and standard protocols.

X-ray Fluorescence images often contain many spectral layers, which can result in long computation times when performing analysis. In this talk, discrete wavelet transformation is used in order to reduce the spectral layers, whilst maintaining the image information so that analysis may be performed at a reduced runtime. Furthermore, once the analysis of the transformed image has been complete, the inverse discrete wavelet transformation is then used to return the image cube back to its original dimensions. The proposed method is effective for cultural heritage as the elemental analysis gathered from XRF images is now obtained in a timely manner.

Implicit information exploration techniques are of great importance for the restoration and conservation of cultural relics. At present, hyperspectral image analysis technique is one of the main methods to extract hidden information, which mainly contains two analysis methods such as principal component analysis (PCA) and minimum noise fraction rotation (MNF), both of which have achieved certain information extraction effects. In recent years, with the development of artificial intelligence, deep learning and other technologies, nonlinear methods such as neural networks are expected to further improve the effect of implicit information mining. Therefore, this paper is oriented to the problem of extracting hidden information from pottery artifacts, and tries to study and explore the hidden information mining method based on deep neural networks, expecting to obtain more stable and richer hidden information. In this paper, an auto-encoder-based implied information mining method is proposed first, and the auto-encoder (AE) framework achieves good performance in feature learning by automatically learning low-dimensional embedding and reconstructing data. However, during the experiments, it is found that some important detailed information (e.g., implicit information) is often lost in the reconstruction process because the traditional autoencoder network only focuses more on the pixel-level reconstruction loss and ignores the overall distribution. Therefore, this paper further proposes a multi-scale convolutional autoencoder network (MSCAE). It constructs a multi-scale convolutional module based on the traditional AE and designs a cyclic consistency loss in addition to the reconstruction loss, so as to reduce the loss of detailed information in the reconstruction process and improve the implicit information mining effect. In the experiments, we find that the proposed method can achieve effective implied information mining by extracting implied information from cocoon-shaped pots and Zhu Shu pottery vases, and its visual effect has been improved compared with the traditional AE network.

We present an automated method for registration and mosaicking of multimodal technical images of artworks based on mutual information. We focus on the registration of element distribution maps resulting from macro X-ray fluorescence (MA-XRF) scanning, which can be considered as a layered stack and treated as the moving image. The target fixed image is the visible image of the same artwork. In consecutive stages, a unique, optimised transformation that provides the highest average mutual information across all images in the stack is identified with consensus. This transformation can be applied to the moving image to obtain the best alignment between the moving and fixed images when overlapped.

Line-field confocal optical coherence tomography (LC-OCT) is an alternative to conventional OCT that combines OCT and confocal microscopy. This technique gives access to three-dimensional (3D) images with a micrometer resolution in the three spatial directions and enhances signal from the deepest layers within the material. After an experimental determination of the device characteristics, the technique is used for the investigation of 18th century gilt leathers from wall-hangings. In these objects, the various layers within the varnish can be identified and the effect of a restoration treatment can be observed to optimize and validate the varnish removal process.

Combining machine learning and physical optical models with advanced portable equipment system, we demonstrate the applicability of a new illumination-induced multispectral imaging system to the examination and detection of forgery in antique polychromic objects.

In this work we present a novel hyperspectral camera based on a compact birefringent interferometer to perform reflectance measurements on works of art. The innovative aspect emphasized in our system lies the capability to acquire hyperspectral images of the sample at different magnification, allowing to image a field of view ranging from few millimetres to tens of centimetres. With our work, we want to demonstrate the effectiveness of this system when performing hyperspectral imaging of painted surfaces at the macro scale, i.e., with few millimetres of field of view, and discuss its potential to resolve the details of a painting as pigments’ grains.

The moir ́e method is a well-known tool in NDT; it is based on the principle of superposition of two sets of lines or gratings, which creates a moir ́e pattern that can reveal surface deformations caused by underlying defects or damage. In this paper, we propose a simple artwork diagnostic using the moir ́e method and a smartphone. The technique is based on coupling the acquisition of fringe patterns by the smartphone camera to an effective fringe generator. The fringe generator consists of a diffractive optical element (DOE) illuminated by a laser diode; this optical device proved to be very effective thanks to its ability to produce in a simple way grid patterns of different spatial frequencies. The smartphone camera is used to capture the grid patterns and to store them in the cloud. We demonstrate the proposed approach by giving some preliminary experimental results.

The colour of the ground layers of a painting shows an influence on the visual appearance of the painting. Optical non-destructive testing (NDT) techniques, for example, optical coherence tomography (OCT), can be used to obtain the characteristics of the details of the paintings non-destructively. The four-flux optical model considers forwards and backwards propagating collimated and diffuse light and is used in this work to investigate the influence of coloured grounds. This paper describes the construction of the model and an evaluation of its performance by comparison with OCT data.

The assessment of the structural condition of cultural heritage objects is important for conservation interventions and their long-term preservation. This investigation concerns The Night Watch (1642), a large-format 17th-century canvas painting by Rembrandt van Rijn that is on display in the Rijksmuseum, Amsterdam. This painting, which has a complex treatment history, has various damaged areas and has undergone three wax-resin relinings. In 1975 the canvas was slashed twelve times with a serrated dinner knife, including several long slashes in the area of Captain Frans Banninck Cocq’s breeches. In 2021, prior to a proposed new structural intervention involving retensioning of the canvas, it was important to evaluate the structural condition of the repaired slashes and of another repair, specifically an old canvas insert in the drum. For this, an in-situ inspection was carried out in the Rijksmuseum as a part of Operation Nightwatch. 3D shearography instrument with thermal loading was used to inspect these two areas of interest on the reverse of The Night Watch. The results showed that the out-of-plane strain in the breeches does not show any large deviations, which alleviated conservators’ concerns about the adhesion of the lining canvas and stability of previous repairs in this region. The patch in the drum showed higher out-of-plane strain variations. This was explained by the lower quality of the patched canvas compared to the repaired slashes in the breeches of Banninck Cocq. Overall, 3D shearography provided valuable inspection results for assurances regarding the structural integrity of the 1975 repairs and the wax-resin lining in The Night Watch, reducing the risks and providing the confidence to proceed with the planned retensioning of the canvas.

Active pulsed thermography was applied for paintworks characterization and under surface defects analysis. Following rapid flash lamp excitation of the samples, a time-resolved thermal response in an extended infrared spectral range was recorded by two high-speed, high-sensitive mid- and long-wavelength infrared cameras. 2D phase shifts images were provided by the FFT software. The method is appropriate for rapid, remote, non-destructive characterization of various defects on painting layers and canvas, and detection of under-drawings, pentimenti, etc. Dynamic multispectral imaging in a wide infrared range may provide complementary information on cultural heritage specimens under inspection for their documentation, preservation and conservation.

The surface is the most representative part of an artwork and is also the part most exposed to alteration due to interaction with the surrounding environment. Non-destructive surface monitoring is of crucial importance in preserving and conserving cultural heritage and optical interferometric techniques allow to acquire the surface structure down to the submicrometric scale In this work, we start from laser microprofilometry based on conoscopic holography sensors to unlock a new way of measuring the surface. In the last years, this technique has proven effective for surface diagnostic in heritage science providing high-quality surface dataset on diffusive, highly reflective, and polychrome artworks. However, an open problem in profilometry is the spatial referencing of surface topography at the micrometer scale, due to the lack of references in the height data with respect to the visually readable surface. We have recently developed a solution that exploits the raw intensity signal collected by the single-point sensor (i.e. the backscattered signal of the laser diode) and the interferometric height dataset, which are intrinsically registered. This method has the potential to be very effective in providing additional information about material texture, color variations or artist's marks that enable spatial registration and data fusion tasks, otherwise difficult in traditional laser profilometry. In this paper we analyzed the feasibility and the performance of the whole process chain from the acquisition to the exploitation of the dual height-intensity datasets, focusing the attention on the raw intensity signal interpreted as a “raw reflectance signal”. We demonstrate the effectiveness of the proposed approach by presenting results on exemplary case studies.

In the engineering field, surface metrology is a valuable tool codified by international standards that enables the quantitative study of small-scale (down to micrometer) surface features, i.e., the surface topography. However, it is not recognized as a resource in heritage science. In literature we find a large use of qualitative inspection of surface morphology or of single-parameter roughness analysis, which confirms the need and potential of such diagnostics. Reasons of the gap are variegate; artworks are hand-made peculiar targets with heterogeneous surfaces, a multiscale approach is necessary, lack of guidelines and unclear meaning of surface roughness descriptors. We propose a critical-constructive discussion through Proof-of-Concept (POC) applications, on the use of surface metrology based on ISO descriptors. Exemplary case studies include: 1) In situ and in-process monitoring of painting microtexture in a Venetian masterpiece: wide and in-band roughness analysis is performed through the complementary use of amplitude, spatial, and hybrid parameters. 2) Multiscale roughness analysis for treatment monitoring in highly reflective metal artworks, requiring high micrometer accuracy in both depth (0.1 μm) and lateral (5μm) directions: surface analysis is performed on scale-limited components to discriminate different surface processes. Surface data are acquired using a prototype of a laser scanning profilometer based on conoscopic holography, with a versatile setup and a surface data pipeline tailored to artwork applications.

The methods and multimodal optomechatronics system for full-field, noninvasive, efficient measurements and monitoring of 3D shape, displacements, strains and spectral content of wide range of cultural heritage objects are presented. The system combines Structured Light method, modified colour 3D Digital Image Correlation and simplified Multispectral Imaging through integrated calibration procedures and data fusion into a common co-ordinate system. The functionality of the system is presented by an example of measurements and monitoring of a historical parchment exposed to changes of relative humidity.

The use of neural encodings is expected to replace the commonly used polynomial fitting in the analysis of artwork surface based on Reflectance Transformation Imaging (RTI), as it has proved to result in more compact encoding with better relight quality, but it is still not widely used due to the lack of efficient implementations available to practitioners. In this work, we demonstrate an optimized system to encode/decode neural relightable images providing interactive visualization in a web interface allowing multi-layer visualization and annotation.

Optical Coherence Tomography (OCT) is often used for non-invasive imaging of paint layers, however, many paint layers are opaque because they are either highly scattering or highly absorbing. In general, light scattering coefficients of paints decrease with increasing wavelength in the near infrared regime (700-2500nm). Therefore, OCT is more likely to be able to penetrate deeper into the paint layers at longer wavelengths, which was why the ISAAC Lab developed an OCT at a longer wavelength of 1960nm than the commonly available ones at 800nm and 1300nm. However, for the same reason that scattering is reduced at longer wavelengths, OCT image contrast also decreases at longer wavelength. In addition, pigments can have strong absorption bands in different regions of the spectral range. Therefore, it is useful to explore OCT at a range of wavelengths for optimum imaging of different materials. While OCT can non-invasively probe the virtual cross-section of paint layers and give some information on the identity of the paint layers based on the optical scattering and absorption properties, it is limited in its ability to identify pigments. Spatially offset Raman spectroscopy at the micro-scales (Micro-SORS) has recently emerged as a viable technique for layer-by-layer pigment identification with high specificity. However, it can only show that one pigment is found below another pigment without information on the precise stratigraphy nor the thickness of the layers. Here we explore the complementarity of the difference wavelength OCT and micro-SORS through examining a mock painting with a four-layer painted stratigraphy that includes i) a wooden support, ii) a preparation layer made of gypsum and animal glue binder with underdrawings, iii) a first paint layer with animal glue binder, and iv) a second paint layer with egg tempera binder. Historical and modern pigments were used on both paint layers. Three in-house developed OCT at 810nm, 1260nm and 1960nm and a defocusing micro-SORS were used in this study.

Within the framework of IPERION-CH, a program that supports the creation of new mobile instrumentation, Laser-induced breakdown spectroscopy (LIBS), laser-Induced Fluorescence (LIF), Raman Spectroscopy and diffuse reflectance spectroscopy were combined to provide simultaneously elemental and molecular complementary information from the same analysis point. The prototype has been implemented for cultural heritage applications on Müstair Monastery mural paintings in the context of Molab European transnational access of IPERION-HS . We demonstrated through previous work the analytical possibilities of this instrumentation on mock-up and several cultural heritage materials and we also described the design and the implementation of the set-up that has been developed taking into account the heritage science constrains. At Müstair Monastery, the first implementation of this new analytical instrumentation on real site has been achieved. This paper will focus on the discussion on the advantages and limitations of this instrumentation in this specific context and the improvement in the development.

Hyperspectral imaging allows material identification using a library of reference spectra and different spectral similarity metrics. In this work, we propose a method for selection of optimal spectral metrics, with an application to classification of historical inks. Hyperspectral images of laboratory and real historical samples are acquired in VNIR and SWIR spectral ranges. Two reflectance libraries are obtained, including eight historical inks. Six spectral similarity metrics are compared, and optimal metrics for classification are selected using three different criteria. Classification results are obtained for laboratory and historical samples.

Taj Mahal, made of exquisite white calcite, continues to deteriorate due to the emission of sulphur dioxide, methane etc. by industries and vehicular exhaust caused by the dense population in the region. Our previous collaborative works on samples with Pietra-Dura works already showed damages and irregularities including surface discoloration due to methane, water inclusions in the volume and sub-surface cracks employing micro-Raman spectroscopy, broadband Terahertz Time Domain Imaging (THz-TDI) and THz Laser Feedback Interferometry (THz-LFI). Here, two types of samples having similar artwork, but one made of marble having high sulphur content have been investigated. Employing energy dispersive X-Ray analysis (EDAX), the sulphur content in the previous calcite sample is found to be nil while the new one has 16% by weight. While visually the samples are similar, under optical and scanning electron microscopy (SEM), calcite presents grainier structure with larger porosity while the other one appears denser with finer porosity. In ultra-low- frequency (ULF) Raman spectroscopy, the calcite sample (less than 0.15% Mg content) produces the significant line at 1100 cm -1 while the marble with sulphur shows markedly different spectral response with significant line at 1010 cm -1 . Using both THz-TDI and THz- CW imaging, we concluded that calcite marble has significantly larger THz penetration even at 1 THz; while the marble with high sulphur content has very low THz penetration even below 0.5 THz and high THz absorption offering higher THz reflectivity. These observations pave the way to objectively detect the extent of environmental damage to marble structures across the globe.

Pulsed infrared thermography is applied to the study of a mold casting Chinese bronze lei 罍 dated to the late Shang dynasty (c.a.1250–1050 BC), currently housed in the Capital Normal University Museum. Many spacers and a defective area of this ancient bronze are partly covered with repair material. By analyzing thermographic images using a one-layer thermal difusion model, it is found that the spacers were specifcally made for this bronze. The thickness of the repairing material in the defective area is measured using thermal quadrupole modelling in multilayer materials. This is the frst application of this method to the feld of cultural heritage conservation. These results provide a deeper understanding of the manufacturing process of ancient Chinese bronzes from the viewpoint of archaeological research. They also help assess the repair status from the conservation viewpoint.

Being one of the most frequently used colour, green colourants not only discolourate but also corrode the paper substrate. However, the degradation phenomena and mechanisms have not been sufficiently studied. In this research, eight valuable objects from 15th – 17th century are selected from the collections in the Maurits Sabbe Library KU Leuven as case studies. Coupled with machine learning methods, narrowband UV-VIS-NIR multispectral imaging (NBMI) and fibre optics reflectance spectroscopy (FORS) are demonstrated as the state-of-the-art techniques to gain an in-depth understanding of the degradation of the green colourants, its effect on the paper substrates, and the preservation needs for the collections.

About three centuries ago Newton discovered that white light is a “mix of rainbow’s colors”. Since then scientists and artists tried to bound the concept of colours into some definition and into collections and catalogues of colours. The results was to have many theories and many ways of classifying colours. The study of human vision and the way our retina works suggested a tristimulus model that began a mathematical model in the ’30 of last century thanks to the work of the CIE (cie.co.au). Although the model evolved in its almost one century of life, the basis are the same of the original one. The model has a couple of huge advantages: it allows to assign to each colour some numeric colorimetric coordinates that can be propagated across any media to “communicate the colour”, and, being based on human physiology, it is able of predicting our perception of colours and colour differences. In the era of digital communication, practically speaking the 100% of images and colours are represented following this model and, if all the colour management chain is correct, we can hardly distinguish the original colour of a flower from its representation on a colour monitor. So one could think that the problem of representing colours is solved. Unfortunately not at all. Reducing a colour to three numbers (the colour coordinates, XYZ or Lab) is an irreversible process, so given XYZ coordinates these could correspond to infinite number of spectra (each of which would give to our eyes the same colour stimulus). This ambiguity generates the metamerism. The spectrum of colour of a flower will be not the same spectrum of the representation of that flower on a monitor, even if they seems to be the same colour. Digital colour imaging fidelity is totally based on the metamerism. We most that we can say of two colours is that they are colorimetrically equal. Colorimetry allows to communicate a stimulus that induces a specific colour in our eyes, it doesn’t communicate the real physical nature of that colour. Every time we need to associate colour to matter, colorimetry become useless. Only a complete spectrum can do that, and this brings us to the need of a spectral imaging approach in all the fields where matter matters. In the first part of this work the authors propose a flexible approach based on custom multibandpass filters and AI calibration that allows to achieve the needed spectral resolution with a tuning of the amount of spectral images used, that will be always less that the classical “one band – one filter” approach. In the second part the spectra obtained from images are compared to spectra measured with laboratory spectrophotometers to establish the reliability of the approach. The third part is a collection of spectral imaging case studies in the field of Cultural Heritage, Architecture and Archeology, where identification of matter was a key factor more than having digital images with colorimetric high fidelity.