Image

Build a Face Recognition System Using FaceNet in Python

The project deals with face recognition using deep learning models. The tools that are used include MTCNN for detecting faces and InceptionResnetV1, which is used to extract face embeddings. The intention is to compare faces, find similarities, and visualize the findings based on cosine similarity.

Project Overview

The project intends to create a system for facial recognition through deep learning techniques. The first step is detecting faces from images using the MTCNN detector. Then the embeddings are generated by the InceptionResnetV1 model for the numerical representation of unique features for each face.

Cosine similarity is used to compare these embeddings to whether two faces belong to different identities. We used a threshold that indicates whether or not two faces are similar. This project also gives the option of searching similar faces within a folder by comparing their embeddings to a reference image. Results are given as visual representations with cosine distance values. This project can be applied in many real-life situations like security, authentication and organizing photo collections.

Technology Stack:

  • Programming Language: Python
  • Deep Learning Framework: FaceNet, Keras
  • Image Preprocessing: OpenCv
  • Face Detection: MTCNN
  • Face Recognition: InceptionResnetV1, Cosine Similarity

Approach

The process begins with detecting faces in images using the MTCNN (Multi-task Cascaded Convolutional Networks) model, which recognizes facial landmarks and positions the face in the image. Once the faces have been detected, the InceptionResnetV1 model generates embeddings for the faces, the condensed numerical representation of every face. These embeddings are high-dimensional captures of relevant facial features to compare with another face or with several. To do that, it calculates the cosine distance between an embedding and one of its neighbors. When the distance goes beyond a specific threshold, these two faces are counted as one. It also allows people to search for similar faces in a folder through the random selection of images, which then get separated through its embeddings with the reference image. Such a process involving face detection, embedding extraction and similarity measurement can create a solid face recognition application that fits into different application setups that would include authentication, security.

Workflow:

  • Data Collection: Collect a dataset containing multiple faces suitable for comparison.
  • Preprocessing: Convert images into the appropriate formats for face detection as well as embedding extraction.
  • Model Selection: Use the MTCNN for face detection and InceptionResnetV1 as embeddings of faces.
  • Cosine Similarity: Use cosine similarity for comparing embeddings and for determining matching faces.
  • Result Presentation: Match and present faces with their cosine distance values to show similarity.
  • Optimization: Tune the threshold for face matching to be improved by different real-world use cases.

Data Collection and Preparation

Data collection

Face Dataset is available in Kaggle. It is possible to conveniently and securely access a Kaggle dataset from within Google Colab after configuring your Kaggle credentials to prevent compromising sensitive information. It brings in the user’s data to collect securely the Kaggle API key and username and assigns them as environment variables. This enables the use of Kaggle’s CLI command (!kaggle datasets download -d atulanandjha/lfwpeople) which authenticates the user and downloads the dataset straight into Colab.

Data Preparation

Data preparation workflow

  • Collected a set of images containing faces.
  • Detected and cropped faces from the images using MTCNN.
  • Converted images to RGB format for model compatibility.
  • Extracted face embeddings from the images using InceptionResnetV1.
  • Stored the processed images and embeddings in an organized directory.

Code Explanation

Step 1

Mount Google Drive

Mount your Google Drive to access and save datasets, models and other resources.

from google.colab import drive
drive.mount('/content/drive')

Installing Libraries

This installs Tensorflow, Keras, Face Recognition, OpenCV, Keras-Facenet and FaceNet-PyTorch. These packages are used to build machine learning models and perform various face recognition tasks.

!pip install tensorflow keras face-recognition opencv-python keras-facenet facenet-pytorch

Face Recognition import modules

The code imports the following libraries for face recognition: OpenCV for image processing, NumPy for array manipulation and MTCNN for face detection. It imports InceptionResnetV1 for face embeddings, SciPy to calculate the cosine distance and PIL to manipulate images.

# Import necessary modules
import os
import cv2
import numpy as np
from facenet_pytorch import MTCNN, InceptionResnetV1
import torch
from scipy.spatial.distance import cosine
from PIL import Image
from google.colab.patches import cv2_imshow

Step 2

Listing Files in Directory

This code lists all the files in the 'lfwpeople_faces' directory. It helps to check the available files in that specific folder.

# List files in the extracted folder
os.listdir('lfwpeople_faces')

Converting Images from .tgz Formats into JPGs

The code extracts images from .tgz files and converts them to JPGs. The images should be converted into RGB mode before being saved in the output directory. It also takes care of handling errors in case there are any errors in the extraction or conversion procedures.

import tarfile
import os
from PIL import Image
def convert_tgz_to_jpg(tgz_file_path, output_dir):
"""
Converts images within a .tgz file to JPG format.
Args:
tgz_file_path: Path to the .tgz file.
output_dir: Directory to save the extracted JPG images.
"""
if not os.path.exists(output_dir):
os.makedirs(output_dir)
try:
with tarfile.open(tgz_file_path, 'r:gz') as tar:
for member in tar.getmembers():
if member.isfile() and member.name.lower().endswith(('.png', '.jpg', '.jpeg', '.bmp', '.gif')): #add other image extensions as needed
try:
f = tar.extractfile(member)
image = Image.open(f)
image_rgb = image.convert("RGB") # Ensure 3 channels
# Construct the output file path
output_file_path = os.path.join(output_dir, os.path.splitext(os.path.basename(member.name))[0] + ".jpg")
# Save the image as JPG
image_rgb.save(output_file_path, "JPEG")
print(f"Converted: {member.name} to {output_file_path}")
except Exception as e:
print(f"Error processing {member.name}: {e}")
except FileNotFoundError:
print(f"Error: .tgz file not found at {tgz_file_path}")
except tarfile.ReadError:
print(f"Error: Invalid .tgz file at {tgz_file_path}")
tgz_file_path = '/content/lfwpeople_faces/lfw-funneled.tgz'
output_directory = '/content/lfwpeople_faces/images'
convert_tgz_to_jpg(tgz_file_path, output_directory)

FaceNet Fine-Tuning Setup

This code sets up a FaceNet-based model for fine-tuning face embeddings. It uses MTCNN for face detection and InceptionResnetV1 as the feature extractor. The model is first frozen, and only the last few layers are unfrozen for fine-tuning with a low learning rate, AdamW optimizer, and MSE loss.

Code Editor