Abstract
This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.
| Original language | English |
|---|---|
| Title of host publication | Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings |
| Publisher | Springer Verlag |
| Pages | 344-354 |
| Number of pages | 11 |
| Volume | 9796 LNCS |
| ISBN (Print) | 9783319458854 |
| DOIs | |
| Publication status | Published - 2016 |
| Event | 38th German Conference on Pattern Recognition, GCPR 2016 - Hannover, Germany Duration: 2016 Sep 12 → 2016 Sep 15 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 9796 LNCS |
| ISSN (Print) | 03029743 |
| ISSN (Electronic) | 16113349 |
Other
| Other | 38th German Conference on Pattern Recognition, GCPR 2016 |
|---|---|
| Country | Germany |
| City | Hannover |
| Period | 16/9/12 → 16/9/15 |
Fingerprint
ASJC Scopus subject areas
- Theoretical Computer Science
- Computer Science(all)
Cite this
Identifying individual facial expressions by deconstructing a neural network. / Arbabzadah, Farhad; Montavon, Grégoire; Muller, Klaus; Samek, Wojciech.
Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings. Vol. 9796 LNCS Springer Verlag, 2016. p. 344-354 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9796 LNCS).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Identifying individual facial expressions by deconstructing a neural network
AU - Arbabzadah, Farhad
AU - Montavon, Grégoire
AU - Muller, Klaus
AU - Samek, Wojciech
PY - 2016
Y1 - 2016
N2 - This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.
AB - This paper focuses on the problem of explaining predictions of psychological attributes such as attractiveness, happiness, confidence and intelligence from face photographs using deep neural networks. Since psychological attribute datasets typically suffer from small sample sizes, we apply transfer learning with two base models to avoid overfitting. These models were trained on an age and gender prediction task, respectively. Using a novel explanation method we extract heatmaps that highlight the parts of the image most responsible for the prediction. We further observe that the explanation method provides important insights into the nature of features of the base model, which allow one to assess the aptitude of the base model for a given transfer learning task. Finally, we observe that the multiclass model is more feature rich than its binary counterpart. The experimental evaluation is performed on the 2222 images from the 10k US faces dataset containing psychological attribute labels as well as on a subset of KDEF images.
UR - http://www.scopus.com/inward/record.url?scp=84988322443&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84988322443&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-45886-1_28
DO - 10.1007/978-3-319-45886-1_28
M3 - Conference contribution
SN - 9783319458854
VL - 9796 LNCS
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 344
EP - 354
BT - Pattern Recognition - 38th German Conference, GCPR 2016, Proceedings
PB - Springer Verlag
ER -