Could Facial Recognition Technology ID the “Girl With a Pearl Earring”?

“Who is it?” is often as big a question for art historians as “Who done it?” The mysterious model of many a famous painting—perhaps none so mysterious as the Girl with a Pearl Earring by Johannes Vermeer (shown above)—can turn many an art buff into an amateur sleuth. Now the amateurs can go pro with the latest in facial recognition technology. Technology developed to catch terrorists can now be used to put (perhaps famous) names to famous faces. According to an article in The Independent, a team at the University of California is doing a “feasibility study” to determine just how well facial recognition software can be used by art historians, and they have a $25,000 NEH grant to help.

FACES (which stands for “Faces, Art and Computerized Evaluation Systems”) consists of an electronic engineer and two art historians at the University of California. Project head and art history professor Conrad Rudolph explains the need for the project in the article. “Before the advent of photography, portraits were, almost by definition, depictions of people who were important in their own worlds,” Rudolph argues. “But, as a walk through almost any major museum will show, a large number of these unidentified portraits from before the 19th century have lost the identities of their subjects.” The premise of FACES is that these famous folk were most likely painted more than once and that the link to their name survives on at least one of those portraits. Create a big enough database of named portraits, they believe, and you’ll begin reuniting portraits with their long-lost monikers.

The premise has some flaws, of course. First of all, you’re assuming that only the famous or infamous appear in such portraits. The novel (and Scarlett Johanssonfilm) based on Girl with a Pearl Earring guesses that the subject of the portrait worked as a maid for Vermeer—a fictional but not implausible theory. In such a case, a named second portrait wouldn’t exist. Secondly, the idea that a named second portrait exists in a searchable form also seems a stretch in both the sense of its pure existence as well as its online existence. Old houses throughout Europe are littered with family portraits in private hands. Who is going to go door to door with a digital camera to build this database? Google?

I had to chuckle a bit, too, at Professor Rudolph’s choice for a portrait test case—Lorenzo de’ Medici. Rudolph admits that his critics have a point that “difficulties are inherent” when comparing portraits simply because of differences in expression, age, pose, lighting, and the individual artist’s style. Painting and sculpture can’t be counted on to have the same fidelity as photography, the medium for which facial recognition was intended. When Rudolph picked “Lorenzo the Magnificent” as his proof that portrait faces rendered by different artists can be recorded as labeled graphs and matched together, he left out one very large detail—Lorenzo’s magnificent schnozzolla. Aside from his patronage for the arts and brutal despotism, Lorenzo de’ Medici was known far and wide for his flat and broad nose. Look at any portrait of “il Magnifico” and you’ll notice the nose first. Look again and you’ll see that the features around it vary from work to work enough to make you wonder which is the “real” Lorenzo. To me, the possibility that the FACES project may be stacking the deck seems as obvious as the nose on its test case’s face.

But what if FACES succeeds? What if we could discover that the Vermeer’s demure earring wearer was a less than virtuous society scandal? What lost duchesses and dukes hide among the familiar portraits of major museums? Such works remain on display for their virtuoso technique, but a name and a story could add an entirely new and fascinating dimension to appreciate. Think of all the clearly taken from life portraits in the religious works of Caravaggio. What meek apostle might bear the ruthless visage of a Renaissance banker? What Virgin Mary’s sweet face began as the tainted image of a fallen woman? Conversely, and just as perversely, what paragon of society virtue ended up as Caravaggio’s Mary Magdalene?

Learning such details would tell us as much about the subject as about the artist. Knowing what might have gone through the mind of Vermeer—that perennial cipher—when he chose his portrait subject would recover much more than just a lost name. I’ll remain dubious over how successful this project will be, but I’ll also remain hopeful that modern technology might allow us to look into the art of the past even deeper.

[Image:Johannes Vermeer. Girl with a Pearl Earring (detail), 1665.]