NEC Tops Competition in Identifying Subjects Wearing Face Masks at DHS 2020 Biometric Technology Rally
An algorithm submitted by NEC Corporation of America (NEC) outpaced all competition in correctly matching images of subjects wearing face masks at the U.S. Department of Homeland Security (DHS) Science and Technology Directorate (S&T) 2020 Biometric Technology Rally. The algorithm submitted by NEC, code-named "Alan" for the anonymized test, achieved the highest True Identification Rate (TIR) at 98.7 percent when combined with acquisition system code-named Glen on subjects wearing a facemask. This TIR result exceeded median performance by as much as 19 percent. The first-place finish in the key category was confirmed in results recently published by DHS S&T.(1)
“Face masks are not a new challenge to NEC. They have been quite commonly used to protect public health throughout Japan and parts of Asia ever since H1N1 in 2009, or even before that,” said Kris Ranganath, CTO of NEC Corporation of America. “We’re pleased that this deep experience allows us to respond to the COVID-19 pandemic quickly to address the hygiene issues with effective contactless technology."
The “focused” face matching category is considered the best test of pure performance of a biometric face matching algorithm as it excludes errors introduced when a front-end system — like a digital camera — cannot acquire a suitable image for processing. Even when including these external errors, NEC’s algorithm still achieved a 95.9 percent TIR, which was a full four percentage points higher than the second-best competitor.
Further, when subjects were not wearing masks NEC’s biometric matching technology was able to meet and exceed the Rally’s goal of “99 percent or better” in each of the simulated scenarios DHS S&T organized. This included a first-place finish for the algorithm code named “Pearl,” with a near-perfect 99.8 percent TIR without acquisition errors, and a 99.3 percent rate when instances of these errors were included. This new set of data reconfirmed earlier tests — including the well-regarded National Institute of Standards and Technology FRVT lab evaluations — that show NEC’s algorithms have been consistently rated the most accurate for more than 10 years.(2)
The Biometric Technology Rally, which took place at the end of 2020 at its affiliated Maryland Test Facility, is an annual event hosted by DHS S&T that allows vendors to demonstrate their technology in front of potential U.S. government buyers. The tests are designed to level the playing field for all products under consideration and follow strict rules. They are done in accordance with government requirements, including testing rules to protect health, safety, and privacy of human subjects.
In 2020, the rally simulated the emerging technical challenge for industry resulting from the coronavirus pandemic: reliably identifying people wearing facemasks. This event consisted of an initial individual processing phase, as well as an advanced group processing phase. Technology providers were required to demonstrate strong performance in the individual processing phase to qualify for the group processing phase.
As part of the test, front-end — or Acquisition Systems — collected biometric images from demographically diverse volunteers. This is basically the physical part of the test that looks at the performance of hardware, like digital cameras, as users are engaged. The images were then processed through back-end Matching Systems, which generally consist of software systems powered by complex biometric algorithms.
Most volunteers had prior experience with biometric devices and process and were told to comply with any instruction but were not briefed on how to use specific products being tested. Groups of volunteers were directed to opt-in or out, then pass through test gates for each product, one at a time, and under a time constraint.
Having acquired images in a simulated real-world scenario, the back-end Matching Systems were fed the images and then evaluated on two factors including: overall algorithm performance and robustness. Overall performance is primarily a test of the algorithm’s effectiveness in making a successful match of a face to an image. Robustness measures how a matching system performs when used with different Acquisition Systems.
“This is one of the best opportunities to see how well facial recognition systems work, how components operate together, and how they fare against each other in similar conditions,” said Benji Hutchison, president of federal operations NEC Corporation of America. “We’re pleased to see that our significant investments in research and development translates to a much better and more reliable customer experience in all sorts of environments.”
1. “Matching System Results for the 2020 Biometric Technology Rally,” published January 27, 2021 by U.S. Department of Homeland Security Science & Technology and the Maryland Test Facility (MdTF).
2. “The Face Recognition Vendor Test 2018 (FRVT 2018),” sponsored by the US Department of Homeland Security and others, assessed the accuracy and speed of authentication for tens of millions of people using face recognition technologies from top vendors from throughout the world.
