Download the VGG16 Face Dataset and the Haar Cascade XML file used for face detection which will be used for the preprocessing in the face recognition task.
faceCascade = cv2.CascadeClassifier(os.path.join(base_path, "haarcascade_frontal_face_default.xml")) # haar cascade detects faces in images
vgg_face_dataset_url = "http://www.robots.ox.ac.uk/~vgg/data/vgg_face/vgg_face_dataset.tar.gz"
with request.urlopen(vgg_face_dataset_url) as r, open(os.path.join(base_path, "vgg_face_dataset.tar.gz"), 'wb') as f:
f.write(r.read())
# extract VGG dataset
with tarfile.open(os.path.join(base_path, "vgg_face_dataset.tar.gz")) as f:
f.extractall(os.path.join(base_path))
# download Haar Cascade for face detection
trained_haarcascade_url = "https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_default.xml"
with request.urlopen(trained_haarcascade_url) as r, open(os.path.join(base_path, "haarcascade_frontalface_default.xml"), 'wb') as f:
f.write(r.read())
Selectively load and process a specific number of images for a set of predefined subjects from the VGG Face Dataset.
# populate the list with the files of the celebrities that will be used for face recognition
all_subjects = [subject for subject in sorted(os.listdir(os.path.join(base_path, "vgg_face_dataset", "files"))) if subject.startswith("Jesse_Eisenberg") or subject.startswith("Sarah_Hyland") or subject.startswith("Michael_Cera") or subject.startswith("Mila_Kunis") and subject.endswith(".txt")]
# define number of subjects and how many pictures to extract
nb_subjects = 4
nb_images_per_subject = 40
Iterate through each subject’s file by opening a text file associated with the subject and reading the contents. Each line in these files contains a URL to an image. For each URL (which points to an image), the code tries to load the image using urllib and convert it into a NumPy array.
images = []
for subject in all_subjects[:nb_subjects]:
with open(os.path.join(base_path, "vgg_face_dataset", "files", subject), 'r') as f:
lines = f.readlines()
images_ = []
for line in lines:
url = line[line.find("http://"): line.find(".jpg") + 4]
try:
res = request.urlopen(url)
img = np.asarray(bytearray(res.read()), dtype="uint8")
# convert the image data into a format suitable for OpenCV
# images are colored
img = cv2.imdecode(img, cv2.IMREAD_COLOR)
h, w = img.shape[:2]
images_.append(img)
cv2_imshow(cv2.resize(img, (w // 5, h // 5)))
except:
pass
# check if the required number of images has been reached
if len(images_) == nb_images_per_subject:
# add the list of images to the main images list and move to the next subject
images.append(images_)
break
Face detection set up
There are 13 convolutional layers, five Max Pooling layers, and three Dense layers. This results in 21 layers with 16 weights, meaning it has 16 learnable parameter layers. VGG16 takes input tensor size as 224x244. The model focuses on having convolution layers of a 3x3 filter with stride 1. It always uses the same padding with a maxpool layer of 2x2 filter of stride 2.
Conv-1 Layer has 64 filters, Conv-2 has 128 filters, Conv-3 has 256 filters, Conv 4 and Conv 5 have 512 filters, and three fully connected layers where the first two have 4096 channels each, the third performs 1000-way ILSVRC classification and contains 1000 channels (one for each class). The final layer is the soft-max layer.
Start preparing the base model.
base_model = VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))
# Set the layers of the base model to be non-trainable
for layer in base_model.layers:
layer.trainable = False
To make sure that the model will classify the images correctly, we need to extend the model with additional layers.
x = base_model.output
x = GlobalAveragePooling2D()(x)
# dense layers
x = Dense(1024, activation='relu')(x)
x = Dense(512, activation='relu')(x)
x = Dense(256, activation='relu')(x)
# add a logistic layer for binary classification
x = Dense(1, activation='sigmoid')(x)
model = Model(inputs=base_model.input, outputs=x)
The Global Average Pooling 2D layer condenses the feature maps obtained from VGG16 into a single 1D vector per map. It simplifies the output and reduces the total number of parameters, aiding in the prevention of overfitting.
The Dense layers are a sequence of fully connected (Dense) layers that are added. Each layer contains a specified number of units (1024, 512, and 256), chosen based on common practices and experimentation. These layers further process the features extracted by VGG16.
The final Dense layer (the Output layer) uses sigmoid activation suitable for binary classification (our two classes being ‘female’ and ‘male’).
Adam Optimization
The Adam Optimization algorithm is an extension of the
Root mean square prop is an adaptive learning algorithm that tries to improve the AdaGrad by taking the “exponential moving average”.
Intuitively, we are adapting to the gradient descent after every iteration so that it remains controlled and unbiased throughout the process, hence the name Adam.
Now, instead of our normal weight parameters mt and vt, we take the bias-corrected weight parameters (m_hat)t and (v_hat)t. Putting them into our general equation, we get:
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.summary()
Set up image data preprocessing, augmentation, and model training in a deep learning context.
train_datagen = ImageDataGenerator()
training_set = train_datagen.flow_from_directory(
'/content/sample_data/deep_learning_assignment/faces/train_augmented',
batch_size=30,
class_mode='binary'
)
validation_datagen = ImageDataGenerator() # used for real-time data augmentation and preprocessing
# generates batches of tensor image data with real-time data augmentation
validation_set = validation_datagen.flow_from_directory(
'/content/sample_data/deep_learning_assignment/faces/valid',
batch_size=30,
class_mode='binary'
)
model.fit(training_set, epochs=10, validation_data=validation_set)
epochs: the number of epochs specifies how much the entire training dataset will be passed forward and backward through the neural network. The model will go through the training data 10 times. An epoch is one complete presentation of the data set to be learned to a learning machine.
batch_size: this parameter defines the number of samples that are propagated through the network at one time. Here, we are using a batch size of 30, meaning the model will take 30 images at a time, process them, update the weights, and then proceed to the next batch of 30 images.
The model’s performance is evaluated by making predictions on the validation set. This gives an idea of how well the model performs unseen data. A threshold is applied to these predictions to classify each image into one of two classes (“male” or “female”).
# Evaluate the model on the validation set
validation_loss, validation_accuracy = model.evaluate(validation_set)
print(f"Validation Accuracy: {validation_accuracy * 100:.2f}%")
print(f"Validation Loss: {validation_loss}")
# Make predictions on the validation set
validation_predictions = model.predict(validation_set)
# Apply threshold to determine class
threshold = 0.5
predicted_classes = (validation_predictions > threshold).astype(int)
# Display the predicted classes along with image names
for i in range(len(validation_set.filenames)):
filename = validation_set.filenames[i]
prediction = predicted_classes[i][0] # Binary predictions, extract single value
class_name = 'male' if prediction == 0 else 'female'
print(f"Image: {filename}, Predicted Class: {class_name}\n")
Create a confusion matrix to visualize the accuracy.
actual_labels = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
predictions = [1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1]
cm = confusion_matrix(actual_labels, predictions)
sns.heatmap(cm, annot=True, fmt='d')
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()
For binary classification, the Receiver Operating Characteristic (ROC) curve and Area Under Curve (AUC) are useful to understand the trade-offs between true positive rate and false positive rate.
fpr, tpr, thresholds = roc_curve(actual_labels, predictions)
roc_auc = auc(fpr, tpr)
plt.figure()
plt.plot(fpr, tpr, color='darkorange', lw=2, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc="lower right")
plt.show()
Conclusion
In conclusion, by using deep learning and image processing algorithms you can build a Python project that recognizes human faces and can categorize them as either male or female.
SOCIAL SHARE CARD GENERATOR