Yuri Kadobnov / Getty Images
Researchers have found a surprisingly simple new method to bypass facial recognition software using makeup models.
A new study from Ben-Gurion University in the Negev discovered that software-generated makeup models can be used to systematically bypass cutting-edge facial recognition software, with makeup applied digitally and physically deceiving some systems with a success rate of up to 98 percent.
In their experience, the researchers defined their 20 participants as blacklisted individuals so that their identification would be reported by the system. They then used a selfie app called YouCam Makeup to digitally apply makeup to facial images based on the heat map that targets the most identifiable areas of the face. A makeup artist then imitated digital makeup on the participants using natural-looking makeup in order to test the target model’s ability to identify them in a realistic situation.
“I was surprised by the results of this study,” Nitzan Guettan, doctoral student and lead author of the study, told Motherboard. “[The makeup artist] didn’t do too many tricks, just saw the makeup in the picture, then tried to copy it to the physical world. It’s not a perfect copy there. There are differences but it still worked.
Researchers tested the attack method in a simulated real-world scenario in which participants in makeup walked down a hallway to see if they would be detected by a facial recognition system. The hallway was fitted with two live cameras that streamed onto the MTCNN face detector while assessing the system’s ability to identify the participant.
“Our attacker assumes a black box scenario, which means the attacker cannot access the target FR model, its architecture, or any of its parameters,” the document explains. “Therefore, [the] the attacker’s only option is to change their face before being captured by the cameras that feed the target FR model.
The experiment was 100% successful in digital experiments on the FaceNet model and LResNet model, according to the document. In the physical experiments, participants were detected in 47.6% of frames if they wore no makeup and 33.7% of frames if they wore randomly applied makeup. Using the researchers’ method of making up the highly identifiable parts of the attacker’s face, they were only recognized in 1.2% of the images.
Researchers aren’t the first to demonstrate how makeup can be used to trick facial recognition systems. In 2010, artist Adam Harvey’s CV Dazzle project showcased a host of makeup looks designed to thwart algorithms, inspired by the “dazzling” camouflage used by warships during WWI.
Various studies have shown how facial recognition systems can be bypassed digitally, for example by create “master faces” who could pass themselves off as others. The article refers to a study where a printable sticker was attached to a hat to bypass the facial recognition system, and another where the spectacle frames have been printed.
While all of these methods can hide someone from facial recognition algorithms, they have the side effect of making you highly visible to other humans, especially if attempted in a highly secure location, such as an airport.
In the researchers’ experiment, they solved this problem by asking the makeup artist to use only conventional makeup techniques and neutral color palettes to achieve a natural look. Given its success in the study, the researchers say this method could technically be replicated by anyone using store-bought makeup.
Perhaps unsurprisingly, Guettan says she generally doesn’t trust facial recognition technology in its current state. “I don’t even use it on my iPhone,” she told Motherboard. “There are a lot of issues with this area of facial recognition. But I think the technology is getting better and better.