Face recognition is a commonly used technology seen in smartphones, laptops, and even televisions. The processes and algorithms I used to achieve a working facial recognition and training program is detailed in this blog.
The first step was to download OpenCV and set it up. Once that was completed, I found a library called “Facerec” by bytefish.de. This library already contains the required algorithms for face recognition. The one I used is called Fisherfaces.
A description of the algorithm is as follows:
- Construct the Imagematrix
X
with each column representing an image. Each image is a assigned to a class in the corresponding class vectorC
. - Project
X
into the(N-c)
-dimensional subspace asP
with the rotation matrixWPca
identified by a Principal Component Analysis, whereN
is the number of samples inX
c
is unique number of classes (length(unique(C))
)
- Calculate the between-classes scatter of the projection
P
as
$latex Sb = \sum_{i=1}^{c} N_i*(mean_i - mean)*(mean_i - mean)^T$
,
where
mean
is the total mean ofP
mean_i
is the mean of classi
inP
N_i
is the number of samples for classi
- Calculate the within-classes scatter of
P
as $latexSw = \sum_{i=1}^{c} \sum_{x_k \in X_i} (x_k - mean_i) * (x_k - mean_i)^T$
, whereX_i
are the samples of classi
x_k
is a sample ofX_i
mean_i
is the mean of classi
inP
- Apply a standard Linear Discriminant Analysis and maximize the ratio of the determinant of between-class scatter and within-class scatter. The solution is given by the set of generalized eigenvectors
Wfld
ofSb
andSw
corresponding to their eigenvalue. The rank ofSb
is at most(c-1)
, so there are only(c-1)
non-zero eigenvalues, cut off the rest. - Finally obtain the Fisherfaces by
W = WPca * Wfld
.
In code, the algorithm looks something like this:
class Fisherfaces(Model): def __init__(self, X=None, y=None, num_components=None, k=1): Model.__init__(self, name="Fisherfaces") self.k = k self.num_components = num_components try: self.compute(X,y) except: pass def compute(self, X, y): """ Compute the Fisherfaces as described in [BHK1997]: Wpcafld = Wpca * Wfld. Args: X [dim x num_data] input data y [1 x num_data] classes """ n = len(y) c = len(np.unique(y)) pca = PCA(X, n-c) lda = LDA(pca.project(X), y, self.num_components) self._eigenvectors = pca.W*lda.W self.num_components = lda.num_components self.P = self.project(X) self.y = y def predict(self, X): Q = self.project(X) return NearestNeighbor.predict(self.P, Q, self.y, k=self.k) def project(self, X): return np.dot(self._eigenvectors.T, X) def reconstruct(self, X): return np.dot(self._eigenvectors, X)
Along with the library was a basic face recognition program from a webcam. However, it only provided a way of recognition from a dataset of cropped images, without an automated way of training faces. With the help of Aurash, I was able to code a method to train people’s faces on the fly. Using a button on the keyboard, the user can take multiple images of themselves at different angles or under different lighting conditions. Once done, the user can exit the program and it will compute the saved images into a model file, which would then be used by the face recognition program.