This article introduces a facial recognition and management system based on deep learning designed by bloggers in detail.
The blog post gives the principle of facial recognition, and at the same time gives the facial recognition implementation code of Python and the UI interface designed by PyQt.
The system realizes a number of functions including
recognizing faces use picture file or mp4 video file
Use the device camera to obtain the real-time picture, and at the same time, you need to recognize the face in the picture. You can also select this function under the “Recognize Face” function option. As shown in the figure below, after clicking the camera button, the system enters the ready state, the system displays the real-time picture and starts to detect the face in the picture.
When a new face appears and needs to be entered, click the “Enter Face” function option button. At this time, the bottom function interface switches to the input function. First enter the face name and click “New” to select the face picture or turn on the camera for picture capture. After the system detects the face, it can select “Take Picture”. The system gets the captured face area picture. Finally, click the “Enter” button to extract all face picture features and write them into the system library.
For the existing facial data, you can select the “Manage Face” function option button, switch to the management interface, select the facial data column to be deleted or updated in the table, and click OK to automatically update the facial data library.
Principles of facial recognition
The application of Dlib in facial recognition: (1) accepting images and loading them into a pixel array for processing; (2) Generate new images using local binary pattern facial descriptions; (3) Write the read image using functions such as scan_image_boxes in the Dlib library, and then calculate the feature vectors between faces; (4) Compare with the feature vectors in the facial database and use the global function threshold_image to calculate the threshold, completing facial recognition 1.
Dlib can be called through Python to implement the process of image preprocessing, feature vector extraction, and verification with data in the face database to determine the identity of a person. Our facial recognition process here includes face detection, face alignment, face representation, and face matching
Steps
The first step is to import several Python dependencies that need to be used. The Python version we use is 3.8.
1 | <span id="1ace" data-selectable-paragraph=""><span>import</span> dlib<br><span>import</span> csv<br><span>import</span> os<br><span>import</span> cv2<br><span>import</span> numpy <span>as</span> np</span> |
We loaded several model methods in dlib. In fact, the code is as follows:
1 | <span id="5f0c" data-selectable-paragraph="">path_face_dir = <span>"./data/database_faces/"</span><br>person_list = os.listdir(path_face_dir)<br>detector = dlib.get_frontal_face_detector()<br>predictor = dlib.shape_predictor('./data/data_dlib/shape_predictor_68_face_landmarks.dat')<br>face_reco_model = dlib.face_recognition_model_v1(<span>"./data/data_dlib"</span><br> <span>"/dlib_face_recognition_resnet_model_v1.dat"</span>)</span> |
Next, we write a function extract_features to extract face features from the image. The function reads the image and uses the face detection machine to obtain the face position, and extracts the features through the Deep Convolutional Neural Network Resnet, and finally obtains a 128-dimensional feature vector. The code is as follows:
1 | <span id="a009" data-selectable-paragraph="">def <span>extract_features</span>(path_img):<br> img_rd = cv2.<span>imdecode</span>(np.<span>fromfile</span>(path_img, dtype=np.uint8), -<span>1</span>)<br> faces = <span>detector</span>(img_rd, <span>1</span>)<br> if <span>len</span>(faces) != <span>0</span>:<br> shape = <span>predictor</span>(img_rd, faces[<span>0</span>])<br> face_descriptor = face_reco_model.<span>compute_face_descriptor</span>(img_rd, shape)<br> else:<br> face_descriptor = <span>0</span><br> return face_descriptor</span> |
Next, we can perform feature extraction, so that we can traverse all the images under each type of face folder in the directory above and extract features, and then take the average value and save it in the csv file to complete the feature extraction and record. The code is implemented as follows:
1 | <span id="a332" data-selectable-paragraph="">with <span>open</span>("./features_all_test.csv", "w", newline="") as csvfile:<br> writer = csv.<span>writer</span>(csvfile)<br> for person in person_list:<br> features_list = []<br> photos_list = os.<span>listdir</span>(path_face_dir + <span>"/"</span> + person)<br> if photos_list:<br> for photo in photos_list:<br> features_128D = <span>extract_features</span>(path_face_dir + <span>"/"</span> + person + <span>"/"</span> + photo)<br> <span>print</span>(<span>"pic"</span> + photo + <span>"recorded！"</span>)<br><br> if features_128D == <span>0</span>:<br> continue<br> else:<br> features_list.<span>append</span>(features_128D)<br> if features_list:<br> features_mean = np.<span>array</span>(features_list).<span>mean</span>(axis=<span>0</span>)<br> else:<br> features_mean = np.<span>zeros</span>(<span>128</span>, dtype=int, order=<span>'C'</span>)<br> str_face = [person]<br> str_face.<span>extend</span>(<span>list</span>(features_mean))<br> writer.<span>writerow</span>(str_face)<br> <span>print</span>(<span>" face record finish！"</span>)</span> |
In the process of matching, we need to calculate the distance (similarity) between the two face feature vectors. Here, we first define an Euclidean distance function euclidean_distance to calculate the feature vectors, and return the distance values of the two vectors:
1 | <span id="a2b7" data-selectable-paragraph="">def <span>euclidean_distance</span>(feature_1, feature_2):<br> <br> feature_1 = np.<span>array</span>(feature_1)<br> feature_2 = np.<span>array</span>(feature_2)<br> dist = np.<span>sqrt</span>(np.<span>sum</span>(np.<span>square</span>(feature_1 - feature_2)))<br> return dist</span> |
In addition, the detected and recognized face results need to be labeled in the image, so a function drawRectBox to add markers is defined here, that is, using OpenCV and PIL to draw markers and text at the face position, the code is as follows:
1 | <span id="998e" data-selectable-paragraph="">def <span>drawRectBox</span>(img, rect_pos, addText):<br> cv2.<span>rectangle</span>(img, (<span>int</span>(<span>round</span>(rect_pos[<span>0</span>])), <span>int</span>(<span>round</span>(rect_pos[<span>1</span>]))),<br> (<span>int</span>(<span>round</span>(rect_pos[<span>2</span>])), <span>int</span>(<span>round</span>(rect_pos[<span>3</span>]))),<br> (<span>0</span>, <span>0</span>, <span>255</span>), <span>2</span>)<br> cv2.<span>rectangle</span>(img, (<span>int</span>(rect_pos[<span>0</span>] - <span>1</span>), <span>int</span>(rect_pos[<span>1</span>]) - <span>16</span>), (<span>int</span>(rect_pos[<span>0</span>] + <span>75</span>), <span>int</span>(rect_pos[<span>1</span>])), (<span>0</span>, <span>0</span>, <span>255</span>), -<span>1</span>,<br> cv2.LINE_AA)<br> img = Image.<span>fromarray</span>(img)<br> draw = ImageDraw.<span>Draw</span>(img)<br> draw.<span>text</span>((<span>int</span>(rect_pos[<span>0</span>] + <span>1</span>), <span>int</span>(rect_pos[<span>1</span>] - <span>16</span>)), addText, (<span>255</span>, <span>255</span>, <span>255</span>), font=fontC)<br> image_x = np.<span>array</span>(img)<br> return image_x</span> |
Use the face detection device to obtain the face position and cut out the face area; then perform feature extraction on the face area and compare it with the features in the library, calculate Euclidean clustering one by one, and find the library face with the smallest distance; compare the minimum distance with the set threshold (0.4). If it is less than 0.4, it means it matches the library face, otherwise it is regarded as an unknown face. The code for this process is as follows:
1 | <span id="e393" data-selectable-paragraph=""><br> image = img_rd.copy()<br> faces = detector(image, <span>0</span>)<br><br> <span>if</span> <span>len</span>(faces) > <span>0</span>:<br> <br> face_feature_list = []<br> face_name_list = []<br> face_position_list = []<br> start_time = time.time()<br><br> <span>for</span> k, d <span>in</span> <span>enumerate</span>(faces):<br> <br> height = (d.bottom() - d.top())<br> width = (d.right() - d.left())<br> hh = <span>int</span>(height / <span>2</span>)<br> ww = <span>int</span>(width / <span>2</span>)<br><br> y2 = d.right() + ww<br> x2 = d.bottom() + hh<br> y1 = d.left() - ww<br> x1 = d.top() - hh<br> <br> <span>if</span> y2 > img_rd.shape[<span>1</span>]:<br> y2 = img_rd.shape[<span>1</span>]<br> <span>elif</span> x2 > img_rd.shape[<span>0</span>]:<br> x2 = img_rd.shape[<span>0</span>]<br> <span>elif</span> y1 < <span>0</span>:<br> y1 = <span>0</span><br> <span>elif</span> x1 < <span>0</span>:<br> x1 = <span>0</span><br><br> <br> crop_face = img_rd[x1: x2, y1: y2]<br> <br> shape = predictor(img_rd, d)<br> face_feature_list.append(face_reco_model.compute_face_descriptor(img_rd, shape))<br><br> current_face = crop_face<br><br> <span>if</span> exist_flag: <br> <span>for</span> k <span>in</span> <span>range</span>(<span>len</span>(faces)):<br> <br> face_name_list.append(<span>"未知人脸"</span>)<br><br> <br> face_position_list.append(<span>tuple</span>(<br> [faces[k].left(), <span>int</span>(faces[k].bottom() + (faces[k].bottom() - faces[k].top()) / <span>4</span>)]))<br><br> <br> current_distance_list = []<br> <span>for</span> i <span>in</span> <span>range</span>(<span>len</span>(face_feature_exist)):<br> <br> <span>if</span> <span>str</span>(face_feature_exist[i][<span>0</span>]) != <span>'0.0'</span>:<br> e_distance_tmp = euclidean_distance(face_feature_list[k],<br> face_feature_exist[i])<br> current_distance_list.append(e_distance_tmp)<br> <span>else</span>:<br> <br> current_distance_list.append(<span>999999999</span>)<br><br> <br> min_dis = <span>min</span>(current_distance_list)<br> similar_person_num = current_distance_list.index(min_dis)<br> <span>if</span> min_dis < <span>0.4</span>:<br> face_name_list[k] = face_name_exist[similar_person_num]<br><br> end_time = time.time()<br> fps_rec = <span>int</span>(<span>1.0</span> / <span>round</span>((end_time - start_time), <span>3</span>))<br><br> <span>for</span> k, d <span>in</span> <span>enumerate</span>(faces):<br> <br> height = (d.bottom() - d.top())<br> width = (d.right() - d.left())<br> hh = <span>int</span>(height / <span>2</span>)<br> ww = <span>int</span>(width / <span>2</span>)<br> rect = (d.left(), d.top(), d.right(), d.bottom())<br> image = drawRectBox(image, rect, face_name_list[k])<br><br> cv2.imshow(<span>'Stream'</span>, image)<br> c = cv2.waitKey(<span>0</span>) & <span>0xff</span></span> |
In addition to the face matching process described above, this part of the code also shows the marking process after recognizing the face. If a face is detected, a rectangular box is drawn according to the unknown coordinates of the face, and the text of the recognition result is added above the rectangular box according to the recognition result. Finally, the marked image is displayed in the window. The running result is as follows:
Remaining is Open the QtDesigner software, drag the following controls to the main window, adjust the interface style and control placement, and design the interface of the facial recognition system.
If you have any questions or need the full code or data or need customized development requirements, you can contact me Email Mail me : slowlon@foxmail.com