Identifying individual facial expressions by deconstructing a neural network

Farhad Arbabzadah, Grégoire Montavon, Klaus Muller, Wojciech Samek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.

Original languageEnglish
Title of host publicationPattern Recognition - 38th German Conference, GCPR 2016, Proceedings
PublisherSpringer Verlag
Pages344-354
Number of pages11
Volume9796 LNCS
ISBN (Print)9783319458854
DOIs
Publication statusPublished - 2016
Event38th German Conference on Pattern Recognition, GCPR 2016 - Hannover, Germany
Duration: 2016 Sep 122016 Sep 15

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9796 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other38th German Conference on Pattern Recognition, GCPR 2016
CountryGermany
CityHannover
Period16/9/1216/9/15

Fingerprint

Facial Expression
Neural Networks
Neural networks
Transfer Learning
Attribute
Prediction
Face
Model
Overfitting
Small Sample Size
Multi-class
Experimental Evaluation
Confidence
Labels
Binary
Subset

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Arbabzadah, F., Montavon, G., Muller, K., & Samek, W. (2016). Identifying individual facial expressions by deconstructing a neural network. In Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings (Vol. 9796 LNCS, pp. 344-354). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9796 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-45886-1_28

Identifying individual facial expressions by deconstructing a neural network. / Arbabzadah, Farhad; Montavon, Grégoire; Muller, Klaus; Samek, Wojciech.

Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings. Vol. 9796 LNCS Springer Verlag, 2016. p. 344-354 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9796 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Arbabzadah, F, Montavon, G, Muller, K & Samek, W 2016, Identifying individual facial expressions by deconstructing a neural network. in Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings. vol. 9796 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9796 LNCS, Springer Verlag, pp. 344-354, 38th German Conference on Pattern Recognition, GCPR 2016, Hannover, Germany, 16/9/12. https://doi.org/10.1007/978-3-319-45886-1_28
Arbabzadah F, Montavon G, Muller K, Samek W. Identifying individual facial expressions by deconstructing a neural network. In Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings. Vol. 9796 LNCS. Springer Verlag. 2016. p. 344-354. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-45886-1_28
Arbabzadah, Farhad ; Montavon, Grégoire ; Muller, Klaus ; Samek, Wojciech. / Identifying individual facial expressions by deconstructing a neural network. Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings. Vol. 9796 LNCS Springer Verlag, 2016. pp. 344-354 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{34c360bb017c4fad994835efb9cfe9f3,
title = "Identifying individual facial expressions by deconstructing a neural network",
abstract = "This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.",
author = "Farhad Arbabzadah and Gr{\'e}goire Montavon and Klaus Muller and Wojciech Samek",
year = "2016",
doi = "10.1007/978-3-319-45886-1_28",
language = "English",
isbn = "9783319458854",
volume = "9796 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "344--354",
booktitle = "Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings",

}

TY - GEN

T1 - Identifying individual facial expressions by deconstructing a neural network

AU - Arbabzadah, Farhad

AU - Montavon, Grégoire

AU - Muller, Klaus

AU - Samek, Wojciech

PY - 2016

Y1 - 2016

N2 - This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.

AB - This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.

UR - http://www.scopus.com/inward/record.url?scp=84988322443&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988322443&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-45886-1_28

DO - 10.1007/978-3-319-45886-1_28

M3 - Conference contribution

SN - 9783319458854

VL - 9796 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 344

EP - 354

BT - Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings

PB - Springer Verlag

ER -