Identifying individual facial expressions by deconstructing a neural network

Farhad Arbabzadah, Grégoire Montavon, Klaus Muller, Wojciech Samek

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.

Original languageEnglish
Title of host publicationPattern Recognition - 38th German Conference, GCPR 2016, Proceedings
PublisherSpringer Verlag
Pages344-354
Number of pages11
Volume9796 LNCS
ISBN (Print)9783319458854
DOIs
Publication statusPublished - 2016
Event38th German Conference on Pattern Recognition, GCPR 2016 - Hannover, Germany
Duration: 2016 Sep 122016 Sep 15

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9796 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other38th German Conference on Pattern Recognition, GCPR 2016
CountryGermany
CityHannover
Period16/9/1216/9/15

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Arbabzadah, F., Montavon, G., Muller, K., & Samek, W. (2016). Identifying individual facial expressions by deconstructing a neural network. In Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings (Vol. 9796 LNCS, pp. 344-354). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9796 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-45886-1_28