DeepFace

Accuracy

DeepFace systems can identify faces with 97% accuracy, almost the same accuracy as a human in the same position. Facebook's facial recognition is more effective than the FBI's technology, which has 85% accuracy. Google's technology, FaceNet is more successful than DeepFace using the same data sets. FaceNet set a record for accuracy, 99.63%. Google's FaceNet incorporates data from Google Photos.

Applications

Facebook uses individual facial recognition templates to find photos that an individual is in so they can review, engage, or share the content. DeepFace protects individuals from impersonation or identity theft. Take, for example, an instance where an individual used someone's profile photo as their own. Through DeepFace, Facebook can identify and alert the person whose information is being misused. To ensure that individuals have control over their facial recognition, Facebook does not share facial templates. Additionally, Facebook will remove images from facial recognition templates if someone has deleted their account or untagged themself from a photo. Individuals also have the ability to turn their facial recognition off on Facebook. If the feature is turned off, Facebook will cease facial recognition for that individual.

Following the release of DeepFace in 2015, its uses have remained fairly stagnant. Because more individuals have uploaded images to Facebook, the algorithm has gotten more accurate. Facebook's DeepFace is the largest facial recognition dataset that currently exists. Because of this, some individuals argue that Facebook's facial ID database could be distributed to government agencies. These uses, however, would be prohibited by most data privacy laws. In response to privacy concerns, Facebook removed their automatic facial recognition feature – allowing individuals to opt in to tagging through DeepFace. This change was implemented in 2019.

Architecture

The DeepFace system consists of four modules: 2D alignment, 3D alignment, frontalization, and neural network. An image of a face is passed through them in sequence, resulting in a 4096-dimensional feature vector representing the face. The feature vector can then be further processed for many different tasks. For example, to identify the face, one can compare it against a list of feature vectors of known faces, and identify the face with the most similar feature vector.

DeepFace uses fiducial point detectors based on existing databases to direct the alignment of faces. The facial alignment begins with a 2D alignment, and then continues with 3D alignment and frontalization. That is, DeepFace's process is two steps. First, it corrects the angles of an image so that the face in the photo is looking forward. To accomplish this, it uses a 3-D model of a face.

2D alignment

The 2D alignment module detects 6 fiducial points on the detected face — the center of the eyes, tip of the nose and mouth location. These points are translated onto a warped image to help detect the face. However, 2D transformation fails to compensate for rotations that are out of place.

3D alignment

In order to align faces, DeepFace uses a generic 3D model wherein 2D images are cropped as 3D versions. The 3D image has 67 fiducial points. After the image has been warped, there are 67 anchor points manually placed on the image to match the 67 fiducial points. A 3D-to-2D camera is then fitted that minimizes losses. Because 3D detected points on the contour of the face can be inaccurate, this step is important.

Frontalization

Because full perspective projections are not modeled, the fitted camera is only an approximation of the individual's actual face. To reduce errors, DeepFace aims to warp the 2D images with smaller distortions. Also, thee camera P is capable of replacing parts of the image and blending them with their symmetrical counterparts.

Neural network

The neural network is a sequence of layers, arranged as follows: convolutional layer - max pooling - convolutional layer - 3 locally connected layers - fully connected layer.

The input is an RGB image of the face, scaled to resolution ${\displaystyle 152\times 152}$, and the output is a real vector of dimension 4096, being the feature vector of the face image.

In the 2014 paper, an additional fully connected layer is added at the end to classify the face image into one of 4030 possible persons that the network had seen during training time.

Reactions

Industry

AI researcher Ben Goertzel said Facebook had "pretty convincingly solved face recognition" with the project, but said it would be incorrect to conclude that deep learning is the entire solution to AI.

Neeraj Kumar, a researcher at the University of Washington said that Facebook's DeepFace shows how large sets of outside data can result in a "higher capacity" model. Because of Facebook's wide access to images of individuals, their facial recognition software can perform better than other software with much smaller data sets.

Media

A Huffington Post piece called the technology "creepy", citing data privacy concerns, noted that some European governments had already required Facebook to delete facial-recognition data. According to Broadcasting & Cable, both Facebook and Google had been invited by the Center for Digital Democracy to attend a 2014 National Telecommunications and Information Administration "stakeholder meeting" to help develop a consumer privacy Bill of Rights, but they both declined. Broadcasting & Cable also noted that Facebook had not released any press announcements concerning DeepFace, although their research paper had been published earlier in the month. Slate said that DeepFace was not being publicized by Facebook because it is wary of another round of headlines decrying DeepFace's creepiness.

Users

Many individuals fear facial recognition technology. The technology's nearly perfect accuracy allows social media companies to create digital profiles of millions of Americans. However, an individual's fear of facial recognition and other privacy concerns does not correspond to a decrease in social media use. Instead, attitudes towards privacy and privacy settings do not have a large impact on an individual's intention to use Facebook apps. Because Facebook is a social media site, individual fears about privacy get over ruled by a desire to participate in social media.

Privacy concerns

BIPA lawsuit

Facebook users raised a class action lawsuit against Facebook under Illinois Biometric Information Privacy Act (BIPA). Illinois has the most comprehensive biometric privacy legislation, regulating the collection of biometric information by commercial entities. Illinois' BIPA requires a corporation that obtains a person's biometric information to obtain a written release, provide them notice that their information is being collected, and state the duration the information will be collected. The lawsuit raised against DeepFace alleges that Facebook's collection of facial identification information for the purpose of the tag suggestion tool violates BIPA. Because Facebook does not give notice or consent to individuals when they use this tool, Facebook users argue that it violates BIPA. The Ninth Circuit denied Facebook's motion to dismiss the case and ultimately certified the case. Facebook sought to appeal to the certification of the Ninth Circuit decision which was ultimately granted. Facebook claims that the case should not have been verified because Plaintiffs have no alleged any harm beyond Facebook's violation of BIPA. Facebook removed their automatic facial recognition tagging feature in 2019, in response to the concerns raised in the lawsuit. Facebook proposed a $550 million settlement to the case, which was rejected. When Facebook increased the settlement to $650 million, the court accepted it. Facebook was ordered to pay their $650 million settlement in early March 2021. 1.6 million residents of Illinois will receive at least $345.

In July 2020, Facebook announced that it is building teams that will look into racism in its algorithms. Facebook's teams will work with Facebook's Responsible AI team to study bias in their systems. The implementation of these programs is recent, and it is still unclear what reforms will be made.

Ten-year challenge

In 2019, a Facebook challenge went viral asking users to post a photo from 10 years ago and one from 2019. The challenge was coined the "10 Year challenge." More than 5 million people participated in the challenge, including many celebrities. Worry arose that Facebook's 10 year challenge was designed to train Facebook's facial recognition database. Kate O'Neill, a writer for Wired, wrote an op-ed that echoed this possibility. Facebook denied that they played a role in generating the challenge. However, individuals have argued that the concerns that underscore theories around the 10 year challenge are echoed by broader concerns about Facebook and the right to privacy.

Racism in facial identification technology

Facial recognition algorithms are not universally successful. While the algorithms are capable of classifying faces with over 90% accuracy in some cases, accuracy is lower when the algorithms are applied to women, black individuals, and young people. The systems falsely identify black and Asian faces 10 to 100 times more than they do with white faces. Because algorithms are primarily trained with white men, systems like DeepFace have a more difficult time identifying them. It is projected that once facial recognition data bases are trained to identify people of color — exposing them to more diverse faces — they will be more successful at identification.