Face recognition is a commonly used technology seen in smartphones, laptops, and even televisions. The processes and algorithms I used to achieve a working facial recognition and training program is detailed in this blog.

The first step was to download OpenCV and set it up. Once that was completed, I found a library called “Facerec” by bytefish.de. This library already contains the required algorithms for face recognition. The one I used is called Fisherfaces.

A description of the algorithm is as follows:

- Construct the Imagematrix
`X`

with each column representing an image. Each image is a assigned to a class in the corresponding class vector`C`

. - Project
`X`

into the`(N-c)`

-dimensional subspace as`P`

with the rotation matrix`WPca`

identified by a Principal Component Analysis, where`N`

is the number of samples in`X`

`c`

is unique number of classes (`length(unique(C))`

)

- Calculate the between-classes scatter of the projection
`P`

as

$latex `Sb = \sum_{i=1}^{c} N_i*(mean_i - mean)*(mean_i - mean)^T$`

,

where

`mean`

is the total mean of`P`

`mean_i`

is the mean of class`i`

in`P`

`N_i`

is the number of samples for class`i`

- Calculate the within-classes scatter of
`P`

as $latex`Sw = \sum_{i=1}^{c} \sum_{x_k \in X_i} (x_k - mean_i) * (x_k - mean_i)^T$`

, where`X_i`

are the samples of class`i`

`x_k`

is a sample of`X_i`

`mean_i`

is the mean of class`i`

in`P`

- Apply a standard Linear Discriminant Analysis and maximize the ratio of the determinant of between-class scatter and within-class scatter. The solution is given by the set of generalized eigenvectors
`Wfld`

of`Sb`

and`Sw`

corresponding to their eigenvalue. The rank of`Sb`

is at most`(c-1)`

, so there are only`(c-1)`

non-zero eigenvalues, cut off the rest. - Finally obtain the Fisherfaces by
`W = WPca * Wfld`

.

In code, the algorithm looks something like this:

class Fisherfaces(Model): def __init__(self, X=None, y=None, num_components=None, k=1): Model.__init__(self, name="Fisherfaces") self.k = k self.num_components = num_components try: self.compute(X,y) except: pass def compute(self, X, y): """ Compute the Fisherfaces as described in [BHK1997]: Wpcafld = Wpca * Wfld. Args: X [dim x num_data] input data y [1 x num_data] classes """ n = len(y) c = len(np.unique(y)) pca = PCA(X, n-c) lda = LDA(pca.project(X), y, self.num_components) self._eigenvectors = pca.W*lda.W self.num_components = lda.num_components self.P = self.project(X) self.y = y def predict(self, X): Q = self.project(X) return NearestNeighbor.predict(self.P, Q, self.y, k=self.k) def project(self, X): return np.dot(self._eigenvectors.T, X) def reconstruct(self, X): return np.dot(self._eigenvectors, X)

Along with the library was a basic face recognition program from a webcam. However, it only provided a way of recognition from a dataset of cropped images, without an automated way of training faces. With the help of Aurash, I was able to code a method to train people’s faces on the fly. Using a button on the keyboard, the user can take multiple images of themselves at different angles or under different lighting conditions. Once done, the user can exit the program and it will compute the saved images into a model file, which would then be used by the face recognition program.