An open view of the Ghent Altarpiece, painted by the Van Eyck brothers in 1432, zoomed in on the Adam and Eve panels (marked in red). [Image: Saint-Bavo's Cathedral, www.lukasweb.be - Art in Flanders; photo Hugo Maertens][Enlarge image]
The subject of seven thefts—not to mention smuggling, attempted arson and other crimes—the Van Eyck brothers’ Ghent Altarpiece (1432) is among history's most celebrated paintings. Nearly six centuries after its completion, researchers are unveiling some of the artwork's best-kept secrets with the help of artificial intelligence (AI).
A U.K.–U.S. research team has successfully deconstructed mixed X-ray images of the 15th-century masterpiece to reveal new details about the two-sided Adam and Eve panels of the famed polyptych painting (Sci. Adv., doi: 10.1126/sciadv.aaw7416). The team, comprising researchers from University College London and National Gallery, U.K., and Duke University, USA, expects that this AI approach to art history will open new doors for investigating, conserving and understanding works of art.
Painting both sides
The Ghent Altarpiece resides in St. Bavo’s Cathedral in Ghent, Belgium; it stands 4.7 meters wide and 3.5 meters tall—roughly the size of a barn door, only priceless. The artwork’s sheer size, complexity and exacting detail have fascinated people since it was painted.
Christian motifs unfurl across 12 intricately painted panels, eight of which are painted on both sides of the three-piece folding construction. This clever architecture allows viewers to see different scenes depending on whether the painting is fully open to display all 12 interior panels or folded in. Somber-looking depictions of Adam and Eve bookend the altarpiece’s open view. These two double-sided panels were the subject of the study’s examinations.
More data, more problems
The double-sided Adam and Eve panels of the painting alongside their mixed X-ray images. [Image: Saint-Bavo's Cathedral, www.lukasweb.be - Art in Flanders; photos: Hugo Maertens (interior view; Adam & Eve), Dominique Provost (exterior view), KIK-IRPA (x-ray).][Enlarge image]
As part of an ongoing conservation project, Belgium’s Royal Institute for Cultural Heritage has collected an abundance of data about the Ghent Altarpiece, including extremely high-resolution digital images from a variety of modern scientific imaging techniques (visible and infrared images, infrared reflectographs, and X-ray images).
X-ray images, in particular, are incredibly informative for art restoration; the method reaches beyond the surface to inform on specific painting techniques, how many paint layers were built up to achieve a certain effect, and even insights into materials construction.
The penetrating nature of the technique, however, can make X-ray images difficult to interpret—especially when an artist has re-used a canvas or has painted on both sides of the material, as is the case for the Adam and Eve panels of the Ghent Altarpiece. All of these layers of information are visibly overlaid and flattened into a single mixed X-ray image.
Divide and conquer
The researchers needed to separate these mixed X-ray images of the double-sided panels into reconstructed images of “one-sided” paintings—a complex image processing challenge.
They devised a convolutional neural network (CNN)-based “self-supervised” scheme for the task. They trained the deep neural network to convert an RGB visual image onto an X-ray image by leveraging the following sources: the visible RGB image of the front panel, the visible RBG image of the back panel and the mixed x-ray image.
They fit a CNN model to take the visual RGB images as input and produce two separated X-ray images as output. The neural network “learned” by minimizing the differences between the sum of the reconstructed X-ray images (the output) and the original mixed X-ray image. According to the team, using this self-supervised approach, it achieved a “near-perfect” separation of the mixed X-ray images.
Art history meets art mystery
According to the researchers, visual inspection of the un-mixed images shows that each image corresponds to just one side of the painting, rather than retaining some of the features from the other side—a major problem with previous state-of-the-art approaches. They view these results as “remarkable” and “unexpected,” and believe this warrants further investigation of the deep neural network’s design.
Going forward, the team suggests further study of the X-ray images to determine if the method has any blind spots. Art historians will also need to examine the separated images in more detail to see if they can glean any new insights from the results.
This type of AI-assisted analysis could prove useful for countless art applications, including material identification, brush-stroke technique analysis and even visualization of hidden designs and underdrawings.