A robust scheme for facial analysis and expression recognition

M. Wallace, S. Ioannou, K. Karpouzis and S. Kollias

Image, Video and Multimedia Systems Lab, School of Electrical and Computer Engineering, National
Technical University of Athens, Athens, Greece

Facial analysis includes a number of processing steps which attempt to detect or track the face, to locate characteristic facial regions such as eyes, mouth and nose on it, to extract and follow the movement of facial features, such as characteristic points in these regions, or model facial gestures using anatomic information about the face.

Due to noise, illumination variations and low resolution capturing devices, the detection of facial features, and consequently, facial feature points, can be inaccurate. For this reason, mechanisms are required that can automatically evaluate the quality of each computed mask, assigning a confidence level to it. The emotion recognition system can take advantage of each feature’s confidence level when analyzing them. Exploitation of anthropometric knowledge, in the form of a set of criteria, evaluating the relation of the extracted features can form such a mechanism.

The detected facial features are used to extract the Feature Points considered in the definition of the Facial Animation Parameters (FAPs), to be used as input features to the recognition system. Each FP inherits the confidence level of the feature from which it derives. FAPs are estimated via the comparison of the FPs of the examined frame to the FPs of a frame that is known to be neutral, i.e. a frame which displays no facial deformations.

A variety of techniques can be used to recognize the underlying emotional states, based on analysis of the FAP features extracted from the user’s face. These include neural network classifiers, clustering techniques, neurofuzzy networks and possibilistic approaches. Techniques that can transfer variations of the FAP variables into rules and adapt these rules to specific user’s characteristics, such as clustering and neurofuzzy methods, or handle uncertainty, such as possibilistic methods are mainly analyzed and used in the paper.

A variety of simulation studies is presented using acted and real life data, which illustrate the good performance of the presented system and of the included techniques.

Paper presented at Measuring Behavior 2005 , 5^th International Conference on Methods and Techniques in Behavioral Research, 30 August - 2 September 2005, Wageningen, The Netherlands.

© 2005 Noldus Information Technology bv