Could Facial Recognition Technology ID the “Girl With a Pearl Earring”?

“Who is it?” is often as big a question for art historians as “Who done it?” The mysterious model of many a famous painting—perhaps none so mysterious as the Girl with a Pearl Earring by Johannes Vermeer (shown above)—can turn many an art buff into an amateur sleuth. Now the amateurs can go pro with the latest in facial recognition technology. Technology developed to catch terrorists can now be used to put (perhaps famous) names to famous faces. According to an article in The Independent, a team at the University of California is doing a “feasibility study” to determine just how well facial recognition software can be used by art historians, and they have a $25,000 NEH grant to help.


FACES (which stands for “Faces, Art and Computerized Evaluation Systems”) consists of an electronic engineer and two art historians at the University of California. Project head and art history professor Conrad Rudolph explains the need for the project in the article. "Before the advent of photography, portraits were, almost by definition, depictions of people who were important in their own worlds,” Rudolph argues. “But, as a walk through almost any major museum will show, a large number of these unidentified portraits from before the 19th century have lost the identities of their subjects." The premise of FACES is that these famous folk were most likely painted more than once and that the link to their name survives on at least one of those portraits. Create a big enough database of named portraits, they believe, and you’ll begin reuniting portraits with their long-lost monikers.

The premise has some flaws, of course. First of all, you’re assuming that only the famous or infamous appear in such portraits. The novel (and Scarlett Johansson film) based on Girl with a Pearl Earring guesses that the subject of the portrait worked as a maid for Vermeer—a fictional but not implausible theory. In such a case, a named second portrait wouldn’t exist. Secondly, the idea that a named second portrait exists in a searchable form also seems a stretch in both the sense of its pure existence as well as its online existence. Old houses throughout Europe are littered with family portraits in private hands. Who is going to go door to door with a digital camera to build this database? Google?

I had to chuckle a bit, too, at Professor Rudolph’s choice for a portrait test case—Lorenzo de' Medici. Rudolph admits that his critics have a point that “difficulties are inherent” when comparing portraits simply because of differences in expression, age, pose, lighting, and the individual artist’s style. Painting and sculpture can’t be counted on to have the same fidelity as photography, the medium for which facial recognition was intended. When Rudolph picked “Lorenzo the Magnificent” as his proof that portrait faces rendered by different artists can be recorded as labeled graphs and matched together, he left out one very large detail—Lorenzo’s magnificent schnozzolla. Aside from his patronage for the arts and brutal despotism, Lorenzo de’ Medici was known far and wide for his flat and broad nose. Look at any portrait of “il Magnifico” and you’ll notice the nose first. Look again and you’ll see that the features around it vary from work to work enough to make you wonder which is the “real” Lorenzo. To me, the possibility that the FACES project may be stacking the deck seems as obvious as the nose on its test case’s face.

But what if FACES succeeds? What if we could discover that the Vermeer’s demure earring wearer was a less than virtuous society scandal? What lost duchesses and dukes hide among the familiar portraits of major museums? Such works remain on display for their virtuoso technique, but a name and a story could add an entirely new and fascinating dimension to appreciate. Think of all the clearly taken from life portraits in the religious works of Caravaggio. What meek apostle might bear the ruthless visage of a Renaissance banker? What Virgin Mary’s sweet face began as the tainted image of a fallen woman? Conversely, and just as perversely, what paragon of society virtue ended up as Caravaggio’s Mary Magdalene?

Learning such details would tell us as much about the subject as about the artist. Knowing what might have gone through the mind of Vermeer—that perennial cipher—when he chose his portrait subject would recover much more than just a lost name. I’ll remain dubious over how successful this project will be, but I’ll also remain hopeful that modern technology might allow us to look into the art of the past even deeper.

[Image: Johannes Vermeer. Girl with a Pearl Earring (detail), 1665.]

Big Think
Sponsored by Lumina Foundation

Upvote/downvote each of the videos below!

As you vote, keep in mind that we are looking for a winner with the most engaging social venture pitch - an idea you would want to invest in.

Keep reading Show less

Why Lil Dicky made this star-studded Earth Day music video

"Earth" features about 30 of the biggest names in entertainment.

Culture & Religion
  • Lil Dicky is a rapper and comedian who released his debut album in 2015.
  • His new music video, "Earth," features artists such as Justin Bieber, Ariana Grande, Ed Sheehan, Kevin Hart, and Leonardo DiCaprio.
  • All proceeds of the music video will go to environmental causes, Dicky said.
Keep reading Show less

After death, you’re aware that you’ve died, say scientists

Some evidence attributes a certain neurological phenomenon to a near death experience.

Credit: Petr Kratochvil. PublicDomainPictures.net.
Surprising Science

Time of death is considered when a person has gone into cardiac arrest. This is the cessation of the electrical impulse that drive the heartbeat. As a result, the heart locks up. The moment the heart stops is considered time of death. But does death overtake our mind immediately afterward or does it slowly creep in?

Keep reading Show less

Behold, the face of a Neolithic dog

He was a very good boy.

Image source: Historic Environment Scotland
Surprising Science
  • A forensic artist in Scotland has made a hyper realistic model of an ancient dog.
  • It was based on the skull of a dog dug up in Orkney, Scotland, which lived and died 4,000 years ago.
  • The model gives us a glimpse of some of the first dogs humans befriended.
Keep reading Show less