Facing the Facts to Keep Our Biometrics Secure

I once transformed my face to look like Ron Swanson — for science. 

I never thought disguising myself with wigs and makeup would be part of my job, but as a NIST facial recognition researcher, I sometimes get to do just that. To make myself look like the gruff character from the show Parks and Recreation, I applied a lot of makeup, a wig and a fake mustache. 

With that look, I could no longer unlock my cellphone with my own face. 

This is an example of what we facial recognition researchers call a “presentation attack.” A presentation attack is often used when someone is trying to look like someone else or not to be seen as themselves. A presentation attacker wants to fake someone else’s face or look like anyone but themselves to avoid being picked up by facial recognition technology.  

While my Ron Swanson experiment was innocent, people thwarting facial recognition technology can have scary results. 

For example, an attacker could copy a person’s identity by using that person’s photo to bypass facial recognition to gain access to the victim’s phone or bank account. 

Another tricky facial recognition technology threat is when someone merges two people’s faces into one photo and uses it to commit identity fraud. This is called “morphing,” and it’s easy to do with various online or mobile tools. It creates real security risks if a person gets a passport with a morphed photo, because it allows multiple people to use the same passport.

Facing Threats With Measurement Science 

Biometrics are physical characteristics often used to identify us, such as fingerprints, iris and face. NIST has been evaluating the performance of biometric algorithms since the 1960s, starting with fingerprints. 

In the facial recognition field, we’ve been evaluating the technology for over 25 years now. 

As facial recognition has become more common — from opening your smartphone to identifying yourself at a national border — our work has become even more high-stakes. 

NIST’s involvement in facial recognition technology is varied. We evaluate the accuracy of facial recognition algorithms and work to understand how to prevent and respond to attacks on facial recognition technology. We’ve also developed standards on how to collect and exchange biometric data used by law enforcement agencies worldwide.

Biometrics: Balancing Security & Convenience

On an ongoing basis, we test various companies’ facial recognition technology. We report how well these algorithms perform on a wide range of imagery, both absolutely and compared to each other. In addition to accuracy, our tests examine factors like how fast the technology works and how much computing power it requires. 

Our work helps companies constantly improve their algorithms, which makes this technology safer and more effective for all who rely on it to keep us safe — whether we realize it or not. 

Additionally, we’ve been examining face morphing detection algorithms. Our hope is that one day, these algorithms can help detect face morphing and prevent identity fraud.

I’ve recently worked on testing methods to identify the presentation attacks I mentioned earlier. Our team recently evaluated images with different types of presentation attacks, ranging from wearing an N95 mask to hide part of the face to holding up a photo of another person to the camera. While some algorithms worked well, none of the algorithms we tested could detect all of the presentation attacks in our test. So, there’s still work to be done in this area. 

Making Facial Recognition Work for Every Face 

We published a report in 2019 about the demographic effects of facial recognition technology. Researchers and others have long expressed concerns about bias in facial recognition. We measured algorithm performance across different demographic groups, including age, race and sex. 

Our research showed many algorithms were less effective in some groups of people than others.

It’s important for us to report these results and make the facial recognition community aware of this problem, so they can work toward solutions. We’ve added demographic accuracy metrics to our “leaderboards,” or regular updates on how well companies’ algorithms are performing. 

I’ve been a researcher in the field of biometrics for over a decade now, and I worked as a government contractor with various agencies before coming to NIST. I started out as a guest researcher here before transitioning to full-time research. 

I love this work because of its impact. This is not abstract research. My work can help thwart hackers or prevent people from getting passports they shouldn’t have. 

Our work is often scrutinized by the public because it is high-profile and important to people’s lives. When I struggled with this in the past, I remember my group leader telling me that the attention is because we’re doing important work. No one would care if the research weren’t relevant. 

That’s what I love about this job: knowing that NIST is using our capabilities and our platform to help make a difference by keeping people safer and combating bias. That feeling of making a difference is really what drives me to keep doing what I do. 

Facing the Future of Biometric Science 

This technology is evolving so rapidly that our work to evaluate it is also always changing. One of the newest projects we’re working on is evaluating age estimation technology. 

Age estimation is a separate area of research from facial recognition because it involves looking at images of one person and estimating their age, rather than comparing images of multiple people to identify a face. 

Recent legislation in some states requires some websites and social media platforms to verify users’ ages, so there has been increasing interest in this technology in recent years. We’ve been working on methods to evaluate these algorithms, just as we do for facial recognition. 

As all things biometric continue to evolve with technology, we at NIST will be ready to test and measure their effectiveness. 

And if that involves me dressing up as another TV character, I’m happy to do that — if it helps our research.