Human super-recognizers see faces better than AI

Randal Reid was arrested by Georgia police on Thanksgiving last year as he was headed to a celebration with his mother. Reid, who is Black, was accused of stealing $10,000 worth of purses in Jefferson Parish, Louisiana, three states over. Police booked him as a fugitive and it wasn’t until he’d spent a week in jail that they realized their error.

Reid wasn’t the guy they were looking for. Police had relied on facial-recognition algorithms and blurry video footage from a store in Metairie, Louisiana to secure warrants for Reid’s arrest. But the algorithm missed a mole on Reid’s face, one of a number of things that led police to release him.

Police and military forces around the world have adopted facial-recognition algorithms to do things like identify criminals and missing children in security and other surveillance footage. But assessments of the technology’s accuracy vary wildly. According to one analysis of eight commercial facial recognition systems, the error rate lies somewhere between 48 and 62 percent, but a 2019 review of 127 facial recognition algorithms by the United States National Institute of Standards Technology found that the best ones make errors just 0.2 percent of the time. At least some of the programs do much worse at identifying darker-skinned faces than white ones, including one that Amazon used to sell to police. (It stopped selling it in 2020.)

We don’t really know why the AI hasn’t caught up to the human yet.

Because of the consequences such errors can have for national security, criminal justice, public safety, and civil rights, police forces in London, Berlin, and Queensland, Australia, have recently begun working with a special set of humans who have a knack for faces. According to James Dunn, a psychologist at the University of New South Wales in Sydney, Australia, they have assembled teams of individuals known as super-recognizers, who are extraordinarily accurate at identifying unfamiliar faces without specific training or experience. Studies suggest superior facial recognition is a skill that is highly heritable, just like its opposite, face blindness, also known as prosopagnosia.

Dunn says super-recognizers are especially good at recognizing faces under certain conditions where AI fails: photos taken outdoors in variable lighting conditions, and blurred or pixelated pictures, such as those taken with a cell phone. “AI is very good at various particular use cases, such as in highly controlled environments and high-quality images,” says Dunn. “For instance, AI is great at borders, where you might use a smart gate with a camera that can scan your passport and determine whether the person wanting to cross is that person in the photograph.” But when you’re looking for one person in a low-resolution image of a crowd, say in closed-circuit television footage, human super recognizers are more accurate. “We don’t really know why the AI hasn’t caught up to the human yet,” he says.

Using humans doesn’t eliminate the problem with racial bias, though. Humans suffer from a well-documented phenomenon called the “other race” effect: We are worse at recognizing the faces of ethnic groups that are less familiar to us. This is true even of super-recognizers. White super recognizers who aren’t familiar with Black faces are better at recognizing Black faces than the average person, but still worse at recognizing Black faces than white faces.

Our goal is to find the Einstein of face recognition.

Training might help: Asian-Australian students who are of Asian descent but grew up in predominantly white communities don’t show “other race” effects for white faces or Asian faces, says Dunn. Presumably, they have spent significant amounts of time studying the faces of people of both races.

Researchers have designed many super-recognizer tests, which differ in terms of the type of stimuli they use—whether it’s just remembering faces over time, or matching a face from a high-resolution photograph to one in a low resolution or ambient photograph. Dunn advises that to identify the true super recognizers it’s best to use multiple tests. The Cambridge Face Memory Test is one researchers frequently use. Dunn’s group has calibrated its own face test to identify the most talented super recognizers.

“Our goal is to find the Einstein of face recognition, that kind of one-in-1-million level talent, that 99.9 percentile kind of person, and see what makes them tick,” says Dunn.

Are you a super-recognizer? Take one of these tests:

The UNSW Face Test (University of New South Wales)

The Cambridge Face Memory Test (Birbeck University of London)

This article originally appeared on Nautilus, a science and culture magazine for curious readers. Sign up for the Nautilus newsletter.