How face masks are fooling facial recognition software

A new study explores how wearing a face mask affects the error rates of popular facial recognition algorithms.

woman with different styles of face masks
Credit: B. Hayes/NIST
  • The study measured the error rates of 89 commercial facial recognition technologies as they attempted to match photos of people with and without masks.
  • Wearing a mask increased error rates by 5 to 50 percent among the algorithms.
  • The researchers said they expect facial recognition technology to get better at recognizing people wearing masks. But it's not clear that that's what Americans want.

Wearing a face mask shields you from not only viruses, but also facial recognition software, according to a new study from the U.S. National Institute of Standards and Technology (NIST).

The study tested how accurately 89 commercial facial recognition algorithms could identify people who had masks on. To do that, the researchers tested a specific method of facial recognition called "one-to-one" matching, where the software compares a photo of a person to a different photo of the same person. The algorithm works by measuring the exact distance between a person's facial features, and it's the same technique used to unlock smartphone and verify passports.

The team tested the algorithms on about 6 million photos. Each photo set showed the same person twice: once with a digitally applied mask, once without. The results showed that masks effectively confused the software, causing a 5 to 50 percent spike in error rate in the algorithms.

Facial recognition experiment

NIST digitally applied mask shapes to photos and tested the performance of face recognition algorithms developed before COVID appeared. Because real-world masks differ, the team came up with variants that included differences in shape, color and nose coverage.

Credit: B. Hayes/NIST

But not all masks thwarted the software equally. For example, black masks led to higher error rates than blue masks (though the researchers said they weren't able to completely explore how color affected the software). Error rates were also higher when people wore wide masks (as opposed to rounder ones) that covered most of the nose.

"With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces," said Mei Ngan, a NIST computer scientist and an author of the report. "We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind."

The researchers said they expect facial-recognition software will get better at recognizing people wearing masks.

"But the data we've taken so far underscores one of the ideas common to previous FRVT tests: Individual algorithms perform differently," Ngan said.

American opinion on facial recognition

But do Americans even want better facial recognition technology? The answer depends on who's deploying the software. A 2019 survey from Pew Research Center found that 56 percent of Americans would trust law enforcement to use facial recognition technology responsibly, while 59 percent said it's acceptable for officials to use the software to monitor public spaces for threats.

Americans are more wary of trusting the private sector with facial recognition. For example, 36 percent of respondents said they'd trust technology companies to use the software responsibly, while only 16 percent said they'd trust advertisers to do the same.

silhouettes of people standing in front of a security cameraTests On Facial Recognition Technology Begun At Berlin Suedkreuz

(Photo by Steffi Loos/Getty Images)

No matter how Americans feel about facial recognition, it's probably here to stay. After all, the FBI already has a database of more than 641 million facial images, many of which simply come from publicly accessible social media posts. And even though cities like San Francisco have banned the technology, police across the country are using it with increasing frequency.

Georgetown Law School's Center on Privacy and Technology estimates that "more than one in four of all American state and local law enforcement agencies can run face recognition searches of their own databases, run those searches on another agency's face recognition system, or have the option to access such a system."


What does kindness look like? It wears a mask.

Northwell Health CEO Michael Dowling has an important favor to ask of the American people.

Sponsored by Northwell Health
  • Michael Dowling is president and CEO of Northwell Health, the largest health care system in New York state. In this PSA, speaking as someone whose company has seen more COVID-19 patients than any other in the country, Dowling implores Americans to wear masks—not only for their own health, but for the health of those around them.
  • The CDC reports that there have been close to 7.9 million cases of coronavirus reported in the United States since January. Around 216,000 people have died from the virus so far with hundreds more added to the tally every day. Several labs around the world are working on solutions, but there is currently no vaccine for COVID-19.
  • The most basic thing that everyone can do to help slow the spread is to practice social distancing, wash your hands, and to wear a mask. The CDC recommends that everyone ages two and up wear a mask that is two or more layers of material and that covers the nose, mouth, and chin. Gaiters and face shields have been shown to be less effective at blocking droplets. Homemade face coverings are acceptable, but wearers should make sure they are constructed out of the proper materials and that they are washed between uses. Wearing a mask is the most important thing you can do to save lives in your community.
Keep reading Show less

Science confirms: Earth has more than one 'moon'

Two massive clouds of dust in orbit around the Earth have been discussed for years and finally proven to exist.

J. Sliz-Balogh, A. Barta and G. Horvath
Surprising Science
  • Hungarian astronomers have proven the existence of two "pseudo-satellites" in orbit around the earth.
  • These dust clouds were first discovered in the sixties, but are so difficult to spot that scientists have debated their existence since then.
  • The findings may be used to decide where to put satellites in the future and will have to be considered when interplanetary space missions are undertaken.
Keep reading Show less

Scientists stumble across new organs in the human head

New cancer-scanning technology reveals a previously unknown detail of human anatomy.

Credit: Valstar et al., Netherlands Cancer Institute
Surprising Science
  • Scientists using new scanning technology and hunting for prostate tumors get a surprise.
  • Behind the nasopharynx is a set of salivary glands that no one knew about.
  • Finding the glands may allow for more complication-free radiation therapies.
Keep reading Show less

Millennials reconsidering finances and future under COVID-19

A new survey found that 27 percent of millennials are saving more money due to the pandemic, but most can't stay within their budgets.

Personal Growth
  • Millennials have been labeled the "unluckiest generation in U.S. history" after the one-two financial punch of the Great Recession and the pandemic shutdowns.
  • A recent survey found that about a third of millennials felt financially unprepared for the pandemic and have begun saving.
  • To achieve financial freedom, millennials will need to take control of their finances and reinterpret their relationship with the economy.
  • Keep reading Show less
    Personal Growth

    6 easy ways to transition to a plant-based diet

    Your health and the health of the planet are not indistinguishable.

    Scroll down to load more…
    Quantcast