This technology enables identification with over 99% accuracy even when the user is wearing a mask, and will be showcased in a cashless Lawson convenience store concept at Fujitsu Technology Square's office in Shin-Kawasaki, Japan ⁽¹⁾, for a field trial beginning in January 2021. Video of the system in use

Users wearing masks pose a challenge to conventional facial recognition filtering technologies, as a large portion of the face is obscured, making authentication impossible in some cases. However, with the newly developed technology, the differences in appearance depending on the presence or absence of a mask can be overcome by generating a composite image that combines the mask with the facial image and testing the model. This achieves a filtering accuracy of over 99%, equivalent to that obtained without a mask. Furthermore, Fujitsu has refined its existing palm vein authentication sensor technology by improving the user interface so users know the optimal distance to position their hand from the sensor for seamless authentication. This technology ranked sixth among 147 participating global vendors and first among Japanese vendors ⁽²⁾ in the facial authentication vendor test conducted by the U.S. National Institute of Standards and Technology (NIST). Development Background:
In recent years, the use of biometric authentication technologies has been on the rise, including in areas such as payment processing in physical stores and ID verification for events. Now, these technologies also promise to help limit the spread of COVID-19 by minimizing human contact during transactions and verification processes. To this end, Fujitsu is accelerating the introduction of contactless user interfaces for authentication by leveraging facial and palm vein recognition, with a series of field trials conducted over the past year (Fig. 1).

Challenges:
In many parts of the world, health authorities have highlighted the use of masks as an important tool to help prevent the spread of infectious diseases such as the COVID-19 virus. However, when facial authentication is performed while wearing a mask, most of the face is obscured, and the accuracy of this method can sometimes be compromised. With this in mind, Fujitsu has worked to improve the accuracy of multi-factor biometric authentication in its pilot store by combining facial filtering technology with palm vein recognition. Furthermore, when considering the system's introduction to general users in physical stores in the future, it will be essential to improve the system's user interface to enhance ease of use, allowing users to register and authenticate without complex explanations or intervention from store employees.
Newly Developed Technology
: Fujitsu Laboratories has developed a data-expanding machine learning technology that enables highly accurate filtering even when the target user is wearing a mask.
The features of the developed technology are as follows.
With typical facial recognition technology, when a user wears a mask, only the exposed eye area is scanned to extract features such as face shape and position. While this method can help reduce the impact of mask use on accurate readings, it cannot extract features from the entire face, raising the possibility that the target user may not be recognized due to insufficient data.
To mitigate this effect, the newly developed technology generates an image of the user's face without a mask, taking into account the extraction of full facial features, such as shape and contours. This means that even when the user is wearing a mask, they can be recognized against the corresponding facial image from a large database with the same level of accuracy as without a mask. Specifically, the technology estimates face position based on facial features such as the position of the eyes and nose. Based on the estimation results, an artificial mask is resized and overlaid on the face image to generate a natural-looking masked face image. In addition, the system uses various types of masks to accommodate situations where users wear masks of different colors, patterns, and shapes (Fig. 2). This makes multi-factor authentication more hygienic and easier to use by allowing people to undergo authentication without removing their masks.
 
Improved User Interface:
The user interface of the palm vein authentication system has also been improved for smoother operation. A palm-shaped light has been added to the sensor, and the color and pattern of the light change depending on the distance of the hand, indicating the correct height for accurate authentication (Fig. 3). This allows people unfamiliar with palm vein authentication to adjust their palms to the correct height for smooth, contactless authentication (Fig. 4).
 
Results:
By using the newly developed technology and user interface, Fujitsu expects to provide safer services and prevent the spread of infectious diseases in various situations, such as payments in physical stores using biometric authentication and personal identification in leisure centers.
Future Plans
: To ensure that users can perform the authentication process themselves using the developed technology and improved user interface, Fujitsu will introduce this technology in a system currently in the verification phase ⁽³⁾ at a Lawson cashless convenience store concept in Shin-Kawasaki Technology Plaza in cooperation with Lawson, a major Japanese retail chain. Fujitsu will conduct verification tests starting January 21, 2021, with the goal of putting the system into practical use by the end of fiscal year 2021.
Related Links
: Demonstration video: "Multi-factor biometric authentication terminal in use"

[1]
A cashierless convenience store concept by Lawson at Fujitsu Technology Square's office in Shin-Kawasaki. Since February 2020, Fujitsu has participated in a proof of concept (POC) conducted by Lawson, Inc. for cashierless solutions at one of its office locations. Based on the POC results, Lawson has been working with Fujitsu, as well as Zippin of the United States, to identify business and technical challenges.
[2]
This technology ranked sixth among 147 participating global vendors and first among Japanese vendors in the facial recognition vendor test conducted by the US National Institute of Standards and Technology (NIST) in November 2020. Effects of the FRVT face mask.
[3]
Fujitsu will introduce this technology in a system currently under verification. This refers to the system provided by VCOGNIZATION TECHNOLOGIES, INC. "Zippin." The system integrates devices such as cameras and weight sensors required for self-checkout, AI functions for customer and product recognition, and functions for linking with payment and inventory management.