Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit

Sunbin Kim, Hyeoncheol Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Facial expression is the most powerful and natural non-verbal emotional communication method. Facial Expression Recognition(FER) has significance in machine learning tasks. Deep Learning models perform well in FER tasks, but it doesn't provide any justification for its decisions. Based on the hypothesis that facial expression is a combination of facial muscle movements, we find that Facial Action Coding Units(AUs) and Emotion label have a relationship in CK+ Dataset. In this paper, we propose a model which utilises AUs to explain Convolutional Neural Network(CNN) model's classification results. The CNN model is trained with CK+ Dataset and classifies emotion based on extracted features. Explanation model classifies the multiple AUs with the extracted features and emotion classes from the CNN model. Our experiment shows that with only features and emotion classes obtained from the CNN model, Explanation model generates AUs very well.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538677896
DOIs
Publication statusPublished - 2019 Apr 1
Event2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Kyoto, Japan
Duration: 2019 Feb 272019 Mar 2

Publication series

Name2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings

Conference

Conference2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019
CountryJapan
CityKyoto
Period19/2/2719/3/2

Fingerprint

Neural networks
Emotion
Network model
Muscle
Learning systems
Labels
Communication
Experiments
Learning model
Justification
Experiment
Deep learning
Machine learning

Keywords

  • Deep learning
  • Explanation Model
  • Facial Action Coding System
  • Facial Expression Recognition
  • Justification

ASJC Scopus subject areas

  • Information Systems and Management
  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems

Cite this

Kim, S., & Kim, H. (2019). Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit. In 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings [8679370] (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BIGCOMP.2019.8679370

Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit. / Kim, Sunbin; Kim, Hyeoncheol.

2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. 8679370 (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kim, S & Kim, H 2019, Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit. in 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings., 8679370, 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019, Kyoto, Japan, 19/2/27. https://doi.org/10.1109/BIGCOMP.2019.8679370
Kim S, Kim H. Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit. In 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. 8679370. (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings). https://doi.org/10.1109/BIGCOMP.2019.8679370
Kim, Sunbin ; Kim, Hyeoncheol. / Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit. 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. (2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings).
@inproceedings{a6d93fb8b921460fbcd91959ed2b46b2,
title = "Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit",
abstract = "Facial expression is the most powerful and natural non-verbal emotional communication method. Facial Expression Recognition(FER) has significance in machine learning tasks. Deep Learning models perform well in FER tasks, but it doesn't provide any justification for its decisions. Based on the hypothesis that facial expression is a combination of facial muscle movements, we find that Facial Action Coding Units(AUs) and Emotion label have a relationship in CK+ Dataset. In this paper, we propose a model which utilises AUs to explain Convolutional Neural Network(CNN) model's classification results. The CNN model is trained with CK+ Dataset and classifies emotion based on extracted features. Explanation model classifies the multiple AUs with the extracted features and emotion classes from the CNN model. Our experiment shows that with only features and emotion classes obtained from the CNN model, Explanation model generates AUs very well.",
keywords = "Deep learning, Explanation Model, Facial Action Coding System, Facial Expression Recognition, Justification",
author = "Sunbin Kim and Hyeoncheol Kim",
year = "2019",
month = "4",
day = "1",
doi = "10.1109/BIGCOMP.2019.8679370",
language = "English",
series = "2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings",

}

TY - GEN

T1 - Deep Explanation Model for Facial Expression Recognition Through Facial Action Coding Unit

AU - Kim, Sunbin

AU - Kim, Hyeoncheol

PY - 2019/4/1

Y1 - 2019/4/1

N2 - Facial expression is the most powerful and natural non-verbal emotional communication method. Facial Expression Recognition(FER) has significance in machine learning tasks. Deep Learning models perform well in FER tasks, but it doesn't provide any justification for its decisions. Based on the hypothesis that facial expression is a combination of facial muscle movements, we find that Facial Action Coding Units(AUs) and Emotion label have a relationship in CK+ Dataset. In this paper, we propose a model which utilises AUs to explain Convolutional Neural Network(CNN) model's classification results. The CNN model is trained with CK+ Dataset and classifies emotion based on extracted features. Explanation model classifies the multiple AUs with the extracted features and emotion classes from the CNN model. Our experiment shows that with only features and emotion classes obtained from the CNN model, Explanation model generates AUs very well.

AB - Facial expression is the most powerful and natural non-verbal emotional communication method. Facial Expression Recognition(FER) has significance in machine learning tasks. Deep Learning models perform well in FER tasks, but it doesn't provide any justification for its decisions. Based on the hypothesis that facial expression is a combination of facial muscle movements, we find that Facial Action Coding Units(AUs) and Emotion label have a relationship in CK+ Dataset. In this paper, we propose a model which utilises AUs to explain Convolutional Neural Network(CNN) model's classification results. The CNN model is trained with CK+ Dataset and classifies emotion based on extracted features. Explanation model classifies the multiple AUs with the extracted features and emotion classes from the CNN model. Our experiment shows that with only features and emotion classes obtained from the CNN model, Explanation model generates AUs very well.

KW - Deep learning

KW - Explanation Model

KW - Facial Action Coding System

KW - Facial Expression Recognition

KW - Justification

UR - http://www.scopus.com/inward/record.url?scp=85064647701&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064647701&partnerID=8YFLogxK

U2 - 10.1109/BIGCOMP.2019.8679370

DO - 10.1109/BIGCOMP.2019.8679370

M3 - Conference contribution

T3 - 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings

BT - 2019 IEEE International Conference on Big Data and Smart Computing, BigComp 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -