Learning-based authentication of Jackson Pollock's paintings
Even apparently "useless" visual features can improve computer-assisted authentication of artwork as long as multiple features are used to train machine classifiers.
The abstract expressionist Jackson Pollock (1912–1956), nicknamed ‘Jack the Dripper,’ is one of America's most important artists. He is best known for his ‘action’ paintings, executed by dripping, pouring, and splashing liquid paint onto horizontal canvases on the floor (see Figure 1). Many artworks of doubtful authorship (and some deliberate fakes) have been generated using this drip technique, however, such as the large, unsigned work Teri Horton purchased for $5.00 in a thrift shop in San Bernadino (CA) in the early 1990s. Although some art scholars have attributed this painting to Pollock, others have contested or rejected this notion. The feature documentary ‘Who the #$&% is Jackson Pollock?’ focuses on this nearly two-decades-old debate, which—once resolved—will determine the work's value at either $5 or $50,000,000.
Art authentication based on signatures, provenance (the documentary record of ownership), chemical studies of media (e.g., oil or acrylic), material studies of support (e.g., paper or canvas), preparation (e.g., sizing), fingerprints, and traditional connoisseurship is not always definitive. Any additional informative objective test could thus be quite valuable.
Physicist and painter Richard Taylor, inspired by the apparent range of scales of structures in Pollock's drip paintings, pioneered the use of fractal-image analysis for authentication of Pollock's works.1 A fractal is a mathematical construct that exhibits self-similarity, such as when parts of the object have nearly the same structure at a range of scales. Taylor estimated the fractal properties of Pollock's works using a box-counting algorithm in which the image is divided into boxes of different sizes, and the fraction of boxes containing any paint of a given color is recorded. The slope of these data points in double-logarithmic space yields the painting's fractal dimension. Taylor and colleagues reported that genuine Pollock paintings generally exhibit a characteristic two-legged shape in this particular 2D space, while fakes do not.
In 2008, physicists Jones-Smith, Mathur, and Krauss criticized Taylor's fractal and box-counting approach.2,3 They suggested that highly artificial images can match the fractal properties of genuine ‘Pollocks’ and asserted that the range of spatial scales the Taylor team used was too small to estimate a true fractal dimension. Finally, they concluded that occlusion of paint layers disrupted estimation of the fractal properties of partially hidden layers. They concluded, therefore, that
“Our data make it clear that the fractal criteria of Taylor et al. should play no role whatsoever in authenticity debates. Given the complete lack of correlation between artist and fractal characteristics that we have found, in particular, the failure of fractal analysis to detect deliberate forgery, it is clear that box-counting data are not useful even as a supplement to other analysis.”3
The debate was thus deadlocked, and fractal-based authentication of Pollock's paintings appeared doomed.
As part of a much broader research program into applying computer vision and pattern recognition to significant problems in the history of art,4,5 we first approached6 the ‘noninformative’ fractal conclusion2,3 theoretically. Figure 2 shows hypothetical distributions of two types of artworks in a visual-feature 2D parameter space, f1×f2, where patterns in categories ω1 and ω2 represent genuine Pollocks and fakes, respectively. Suppose that f1 represents some fractal property. When projected onto the f1 axis, both data sets exhibit the same distributions. Thus, this feature—taken alone—does not provide any discriminative information and is therefore ‘useless.’ However, when used in conjunction with another feature (e.g., f2), f1 can potentially be useful.
Second, we pointed out6 that while criticism about the limited range of spatial scales may be valid, it is irrelevant to the problem of pattern recognition, at least in theory. In pattern recognition, a feature need not conform to some specified mathematical form; it needs merely to be specified in an objective, repeatable way. Third, we also pointed out6 that—for Pollock paintings—a statistical classifier does not need to recover the properties of the actual drip patterns hidden by higher layers. The information must simply be useful in a given classifier, a matter that can be assessed only empirically.
Our most important results,6 however, were empirical. We trained the standard Perceptron and nearest-neighbor classifiers to distinguish genuine from fake Pollocks using fractal information and four other features (i.e., Levy dimension, genus, and two features based on oriented energy).7 While the fractal feature alone provided only slightly better than chance (52.4%) accuracy for one feature and the other features resulted in 76.2%, a classifier trained to use all five features yields 81.0% accuracy. Clearly, fractal-based features can be of some use. We believe that our results justify exploring such next steps in Pollock-authentication studies.
Although our recent results appear to break the deadlock in the debate, despite a number of shortcomings, much work must be done before we can provide robust assistance to art scholars. Our empirical results were based on a small set of data and we used admittedly poor ‘fakes’ of somewhat low resolution. We need more image data of higher resolution, and must both explore better features and use sophisticated machine-learning techniques such as boosting, bagging, and cross-validation.7 Nevertheless, we are confident that we are operating on a firm theoretical foundation and that we have enough empirical evidence to suggest that our ongoing research is headed in the right direction towards providing real value to the art community.
I would like to thank Richard Taylor and Katherine Jones-Smith for helpful discussions and the Pollock-Krasner Foundation/Artists Rights Society for permission to reproduce Pollock's Convergence.
David Stork is chief scientist of Ricoh Innovations and consulting professor at Stanford University. He has published six books and proceedings volumes, including Seeing the Light, Pattern Classification (2nd ed.), and Computer Image Analysis in the Study of Art. He holds 38 US patents and has published over 130 technical papers. He is a fellow of the International Association of Pattern Recognition and has lectured at numerous museums worldwide.