Information Sciences Institute

Biometrics systems like iris scanning or fingerprint recognition are no longer fantastical concepts of a distant sci-fi future. From iPhone fingerprint locking to facial authentication systems in airports, it’s safe to say that the future is very much right now.

“It could be reasonable to claim that using automatic biometric-based authentication has become a way of life,” said Mohamed Hussein, computer scientist at USC’s Information Sciences Institute (ISI). “Biometric authentication offers major advantages when it comes to securing our personal devices”, he said, noting that these systems are also more cost-effective. “As these advantages make more people and institutions want to adopt the technology, they also allure some people to find holes in the technology, sometimes just for the sake of amusement.”

To combat those that look for these “holes in the technology,” ISI researchers in the Vision, Image, Speech, and Text Analysis (VISTA) group worked on the Biometric Authentication with Timeless Learner (BATL) project, in which researchers develop systems and algorithms resistant to spoofing of biometric authentication systems. As more people are opting to use biometric technology, the VISTA group’s research focuses on securing these systems, ensuring that outsiders won’t be able to take someone’s photo or fingerprint and use it to break in to their personal devices.

After development, the VISTA group’s biometric systems underwent rigorous third-party testing over a three-week period at the prestigious John Hopkins Applied Physics Laboratory (JHU APL), where various spoofing attacks were presented to try to break into the systems.

The results were impressive. The third-party testing at APL determined that the facial biometric system had a 100% accuracy rate, meaning the system didn’t make a single mistake in identifying biometric attacks over the three weeks of testing. Also, an overall of 99.36% accuracy rate for the iris recognition system and a 99.08% accuracy rate for the fingerprint recognition systems, both higher figures than expected.

Senses of Security

The BATL project was funded under the Intelligence Advanced Research Projects Activity (IARPA) Odin program to strengthen biometric security systems. “The BATL project’s goal is to create novel sensing hardware for face, fingerprint and iris biometrics that provide unconventional data sources,” said Wael AbdAlmageed, PI of the project and research associate professor at USC Viterbi’s Department of Electrical and Computer Engineering. “Our aim is to develop state-of-the-art machine learning and artificial intelligence algorithms that can quickly and accurately detect biometric attacks on iris, facial, and fingerprint recognition systems.”

Before this project began, a typical face recognition biometric system would use a regular DSLR camera to capture your facial data. In fact, if you approached that camera with a photo of yourself, it wouldn’t be able to tell the difference between the 2D photo and the 3D you. As part of the Odin project, the VISTA team’s goal was to develop systems that could differentiate between a real, authentic person and various spoofing attacks used to break into biometric systems.

“As advantages make more people want to adopt [biometrics], they also allure some to find holes in the technology, sometimes just for the sake of amusement.”
Mohamed Hussein

The project consisted of developing both hardware and software. The first step for the team was to make hardware systems for face, iris, and fingerprint recognition. They built two different physical replicas of the biometric systems, one of which was used at the test facility at APL.

“We had to make sure that both systems were as similar as possible to achieve the best performance,” said Leonidas Spinoulas, computer scientist at ISI. “Since our systems are experimental prototypes and built in the lab, this involved a lot of trials where data would be collected from all cameras and [we’d] try to ensure consistency between the two systems.”

A biometric sample, such as a face, iris, or fingerprint, is analyzed by the system using multispectral data, which involves shining LED lights with different wavelengths (visible, near-infrared, long-wave-infrared, etc.) on the sample. A synchronized sequence between the cameras and LEDs makes sure that the most amount of data can be recorded in the least amount of time.

The second step was developing the software. The cameras don’t fully solve the issue—they provide a stream of data, but the data needs to be analyzed so the systems can determine if the biometric sample (the iris, face, or fingerprint) is real or fake. So the team created new machine learning algorithms to analyze the collected data. The algorithms help identify whether the system is dealing with a real person or a mask; a real fingerprint or a piece of silicone; a contact lens/glass eye or a real eye.

In January, the VISTA team sent their biometric systems to APL for a third-party evaluation. Halloween came early at APL, as real live Presentation Attacks (PAs), or fake samples, were produced, such as people wearing masks, fake fingerprints, contact lenses, and other spoofs to see if the BATL team’s biometric systems could be broken into.

By the end of the three-week testing period, data was collected from about 700 participants. The analysis found that for fingerprint and iris recognition, the systems were over 99% accurate in detecting spoofs, while the facial recognition testing system didn’t make a single error.

Iris and Facial PAD Systems

Spinoulas was responsible for the iris classification pipeline for the BATL project. He used several iris extraction methods that examined different areas of the iris (e.g., considering only the iris region, considering both the iris and the pupil, etc.) to study how the different methods could benefit the system’s performance.

One of the constraints of biometrics can be dealing with not enough data when designing algorithms. “This was aggravated in our case, since, throughout the period of the BATL project, we changed decisions on our hardware multiple times, so the earlier data wouldn’t be 100% consistent with data captured during the latest data collections,” said Spinoulas. This led him to come up with different approaches on how data from inconsistent sources can be used and how to avoid errors when training the algorithms.

The VISTA group also worked with collaborators for the facial Presentation Attack Detection (PAD) system. “Our best performing face PAD module was developed by our collaborators at the Idiap Research Institute in Switzerland,” Hussein said. “[The approach] encompasses a novel loss function to enable detecting never-seen-before face presentation attacks.”

Hussein said that the sensor suites used a range of both conventional and unconventional sensing modalities, which are sensors that measure different types of energy, such as light energy reflected by the skin or thermal energy emitted from the skin. These sensing modalities are similar to human senses: touch, sight, hearing, smell, and taste. “The idea is that attackers can’t fool all sensing modalities at the same time, which naturally leads to higher accuracy of our system,” he continued. “On the software side, our machine learning models are evaluated in many different ways to make sure our final models are resilient to different types of attacks.”

Fingerprint PAD System

Hengameh Mirzaalian, a machine learning and computer vision scientist at ISI, was part of the BATL project related to fingerprint PAD, which detects fake fingerprints.

“Fingerprint PAD is an increasingly challenging problem due to the continuous advancement of attack techniques, which generate ‘realistic-looking’ fake fingerprint presentations,” explained Mirzaalian. Similar to the other systems, instead of only relying on existing fingerprint images used within the research community, the team utilized other sensor modalities so they could measure different aspects of the finger, such as blood motion, vein visualization, and the amount of light penetration.

To ensure the fingerprint PAD system would be accurate, the VISTA team collected a large fingerprint dataset from a variety of skin colors, ages, genders, and races, including many PA samples. “This task is absolutely important, and I’d say one of the most important contributions of this project is releasing our collected dataset, which can be used by people in the fingerprint PAD community,” Mirzaalian said. With their collected data in tow, the team developed deep neural networks that can predict the presence or absence of an attack on a test image.

With fingerprint systems being applied in more industries for heightened security, the BATL group’s research is important for any agency, organization, or company that’s applying legacy fingerprint systems, like airports, Mirzaalian said. “The nice thing about our fingerprint station is that it’s touchless, making it less vulnerable to fingerprint lifting and is more desirable from the hygiene perspective,” she added. “[This] is a super valuable feature in this COVID-19 era.”

Published on June 15th, 2020

Last updated on June 16th, 2020