A Heterogeneous Face Recognition System

Introduction

This is a command-line implementation of [1]. Database is CUHK student data set.
According to [1], this algorithm can be used in a variety of heterogeneous face, including near-infrared and thermal infrared, although only sketch face database is used to test.

Run

Environment

This system requires a MATLAB™ environment. Toolbox listed below is required:

• MATLAB™ Image Processing Toolbox
• MATLAB™ Computer Vision System Toolbox

In image normalization step, a valid Microsoft Azure Cognitive Services Computer Vision API subscription is required for eye detection.

Run the entire process in command line

Run

>> main

in MATLAB Command Window.

Run in the GUI

A ugly GUI is implemented in sketch.mlapp with MATLAB native GUI library.

Details of each step

Load training data

1. Normalize each image

   Technical details
   

   1. This system use Microsoft Cognitive Service to detect eye position of the face, and normalize to a standard 200 * 250 pixels image. Because Microsoft Cognitive Service is a RESTful API, Internet connection is required at this step. It's worth to mention that if the input image size is already 200 * 250 pixels, normalization will not be applied.
   2. Related code of eye detection can be seen in eyeDetection.m.
   3. After eye detection, a cropping is applied to the image to divide the image into small patches.
   4. Related code of this step is in train.m.

2. Extract all features from each image

   Technical details
   

   • The motivation of this step is to speed up the training step and testing step.
   • According to [1], 3 filters and 2 feature descriptors are applied to the image, which are Difference of Gaussian (dog), Center-Surround Divisive Normalization (csdn), Gaussian smoothing filter (gaussian) and Scale-invariant Feature Transform (SIFT), Modified Local Binary Patterns (MLBP), respectively.
   • Related function is featureExtraction.

Training

Use train to train data, and return a discriminant matrix W using LDA.

Technical details
As mentioned above, we use CUHK for training. More specificly, 88 faces were used for training, which is cropped. Our cropped algorithm wasn't applied to face images because there are already cropped sketches and photos in CUHK dataset.

Load testing data

In this step, testing gallery was loaded and features were extracted. The motivation of this step is as same as the first step.

Testing

In this step, function testing was applied to the testing probe set, to get the recognition result.
100 images in the database were used for testing the accuracy of the algorithm.

Technical details
In this step, prototype random subspace (P-RS) and direct random subspace (D-RS) are both used. The final score is the sum of P-RS score and D-RS score. See [1] for details.

Performance

Accuracy

Database	Accuracy
CUHK testing databse	85%1,3
PRIP	27.5%1,4

Rank-N accuracy

Note:

1. Due to the random subspace approach, this accuracy result can't be promised in each experiment.
2. Each experiment are trained by CUHK training dataset.

Time Performance

CPU	Average Time(seconds/sketch)
Intel® Core™ i5-4200M 2.50GHz	4.711
Intel® Xeon® E5-2673 v3 2.40GHz	2.202

1: P-RS + D-RS
2: Only P-RS
3: Rank-1 accuracy
4: Rank-5 accuracy

Differences from the paper

1. The combination SIFT and DoG are abandoned due to an all zero feature descriptor vector it will cause.
2. In score level fusion, only scores using the same descriptor were added. See section 5.4 in [1].

Library used

• VLFeat, a computer vision algorithm implementation on MATLAB. vl_sift is used to extract SIFT feature descriptors from images.
• LDA, a LDA implementation on MATLAB.
• RotateAround, a MATLAB function to rotate a image around a specfic point.

FAQ

1. VCOMP100.DLL missing

   When use vl_feat, MATLAB shows

   Missing dependent shared libraries:
   'VCOMP100.DLL' required by ...
   

   There is a solution to the problem.

Reference

[1] B. F. Klare and A. K. Jain, "Heterogeneous face recognition using kernel prototype similarities," IEEE Trans Pattern Anal Mach Intell, vol. 35, no. 6, pp. 1410-22, Jun 2013.