MindSpore: Know who you are without taking off your mask

[Recommended topics in this issue] Starting from the three mainstream front-end technologies, look at their respective characteristics, and how to choose a suitable framework based on business characteristics.

Abstract: We designed a face recognition algorithm based on MindSpore to solve the problem of face recognition in scenes occluded by masks. The open source code of the algorithm has been released on MindSpore.

This article is shared from the Huawei Cloud Community " Know who you are without taking off the mask, MindSpore trains the mask face recognition model ", the original author: chengxiaoli.

The goal of face recognition is to determine the identity of the face image. With the construction of smart cities, the application scenarios of face recognition are becoming more and more diverse, such as suspect tracking and locating missing elderly and children. With the global outbreak of the new coronavirus, more and more people wear masks in public places to reduce the risk of contracting infectious diseases for public health safety.

Traditional face analysis methods mainly extract the global features of the entire face for analysis. In the mask occlusion scene, some important parts of the face (such as the lower jaw) are covered by the mask. Extracting the global features of the entire face will introduce a lot of noise . Therefore, the face analysis task in the mask occlusion scene urgently needs a new solution. We designed a face recognition algorithm based on MindSpore, to solve the face recognition problem in the mask occlusion scene. The open source code of the algorithm has been released on MindSpore.

• paper:

Feifei Ding, Peixi Peng, Yangru Huang, Mengyue Geng and Yonghong Tian. Masked Face Recognition with Latent Part Detection. ACM Multimedia 2020. https://dl.acm.org/doi/pdf/10.1145/3394171.3413731

• Open source address: https://gitee.com/mindspore/mindspore/tree/master/model_zoo/research/cv/MaskedFaceRecognition

Algorithm framework

Currently, there is no large-scale training data set for mask faces. To solve the problem of insufficient training data, we generate mask face data as training data based on the existing public face data set. The process is as follows:

(1) Use existing face key point detection algorithms (such as Dlib) to detect multiple key point positions on unobstructed face images;

(2) Mark the position of key points in the area where the exit mask is worn (the area below the tip of the nose);

(3) Manually mark the key points on the mask example picture (such as N95 mask), and correspond to the key points in the area where the face mask is worn one by one;

(4) Triangulate the mask example picture according to the key point positions of the mask example picture, and divide the mask example picture into multiple small triangles;

(5) According to the key point correspondence between the mask example picture and the face picture, each small triangular area on the mask example picture is affinely transformed to the face picture, and smoothing operation is performed to generate the mask face picture.

We generated 8 different styles of mask faces on the public face dataset WebFace, and mixed them with the original pictures of faces without masks as training data.

The algorithm framework is shown in the figure. Considering that many important information in the face of the mask is occluded by the mask, we propose a two-branch network model based on latent region detection to extract more discriminative features. Among them, the local branch extracts local features from the potential region, and the global branch extracts global features from the original image.

Experimental results

We collected the actual mask face data set PKU-Masked-Face Dataset as a test set. The test set has a total of 10301 face images belonging to 1018 different identities. Most identities have at least 5 faces with masks and 5 faces without masks, totaling 5 angles of view. The 5 angles are: front, left, right, looking down, and looking up. Mask face pictures are used as the query image database, and normal face pictures are used as the image database to be matched.

Test set link:
https://pkuml.org/resources/pku-masked-face-dataset.html

The experimental results of the algorithm model based on MindSpore are shown in the following table. Baseline is the ResNet50 benchmark model trained with WebFace raw data, MG is the ResNet50 model trained with WebFace enhanced data, and LPD is our potential area detection model.

MindSpore code implementation

Second branch network structure code:

Among them, global_out and partial_out correspond to the global feature extracted from the original image and the local feature extracted from the latent region, respectively, and the two parts share the feature extraction backbone layer.

The specific implementation of the potential area detection LPD module:

The input is the original picture, and the output is the boundary coordinates of the unoccluded face area.

• Related training and inference code:

https://gitee.com/mindspore/mindspore/tree/master/model_zoo/research/cv/MaskedFaceRecognition

The code is implemented based on the MindSpore framework, and the hardware environment it runs is Ascend910. The algorithm solves the problem of face recognition in the mask occlusion scene, and significantly improves the performance of the benchmark model. The experimental results are expressed as above to the industry's leading level.

Click to follow and learn about Huawei Cloud's fresh technology for the first time~