Facial Expression Recognition

  • Sarah Eldafrawy
  • Mostafa Labib

Problem Statement

Facial Expression Recognition (FER), as the primary processing method for non-verbal intentions, is an important and promising field of computer vision and artificial intelligence, and one of the subject areas of symmetry. Behaviors, actions, poses, facial expressions and speech; these are considered as channels that convey human emotions. Extensive research has been carried out to explore the relationships between these channels and emotions. This proposal proposes a system which automatically recognizes the emotion represented on a face. Thus a neural network based solution combined with image processing is used in classifying the universal emotions.

Dataset

FER2013 Face Dataset It includes 35,887 face images wild and spontaneous not pre-defined simulating real world pictures taken. Also, it contains images taken in a fixed position and pre-prepared environment. The dataset state of the art using CNN approach is 72.1%.

Input/Output Examples



State of the art



Orignial Model from Literature



Proposed Models







Results

The best model we proposed is the Resnet model it overcomes two of the state of the art models with Accuracy 62%. The two other models was out of the competition







Technical report

  • We used Python Keras to build, train and test our models. We also used numpy, opencv, matplotlib and scipy for utils purposes
  • We used colab with tpu and azure
  • Resnet takes about 2.5 hours to finish training
  • Number of epochs: 15
  • Time per epoch: about 10 minutes
  • Inceptionresnet needs high computational power so in order to test it in feasible time we used part of the dataset

Future work

  1. Continue hyperparameters tuning.
  2. Optimize Resnet Inception model.
  3. Provide needed processing power to reach better accuracy.
  4. Try other datasets to address the FER problem.

Lessons learned & interesting findings

  1. Don’t try to re-implement model keras already implemented, it will be much worse
  2. Validation plots were misleading most of the time

References

List all references here, the following are only examples

  1. Cubic SVM+HoG
  2. CNN
  3. CNN(DeeperCNN)
  4. Dataset URL
  5. Facial Expression Recognition: A Survey