Classification of faces in man and machine

Arnulf B A Graf, Felix A. Wichmann, Heinrich Bulthoff, Bernhard Schölkopf

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

We attempt to shed light on the algorithms humans use to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. First, we apply principal component analysis (PCA) on the pixel information of the face stimuli. We then obtain a data set composed of these PCA eigenvectors combined with the subjects' gender estimates of the corresponding stimuli. Second, we model the gender classification process on this data set using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning: the support vector machine (SVM), the relevance vector machine, the prototype classifier, and the K-means classifier. The classification behavior of humans and machines is then analyzed in three steps. First, the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects' internal decision boundary by studying the training errors of the machines. Second, we study the correlations between the rank-order of the subjects' responses to each stimulus - the gender estimate with its reaction time and confidence rating - and the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects' gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans. Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based on stimuli close to the boundary between the classes.

Original languageEnglish
Pages (from-to)143-165
Number of pages23
JournalNeural Computation
Volume18
Issue number1
DOIs
Publication statusPublished - 2006 Jan 1
Externally publishedYes

Fingerprint

Classifiers
Principal component analysis
Support vector machines
Learning systems
Principal Component Analysis
Eigenvalues and eigenfunctions
Psychophysics
Stimulus
Pixels
Patient Selection
Reaction Time
Classifier
Datasets
Support Vector Machine
Machine Learning
Prototype

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Artificial Intelligence
  • Neuroscience(all)

Cite this

Graf, A. B. A., Wichmann, F. A., Bulthoff, H., & Schölkopf, B. (2006). Classification of faces in man and machine. Neural Computation, 18(1), 143-165. https://doi.org/10.1162/089976606774841611

Classification of faces in man and machine. / Graf, Arnulf B A; Wichmann, Felix A.; Bulthoff, Heinrich; Schölkopf, Bernhard.

In: Neural Computation, Vol. 18, No. 1, 01.01.2006, p. 143-165.

Research output: Contribution to journalArticle

Graf, ABA, Wichmann, FA, Bulthoff, H & Schölkopf, B 2006, 'Classification of faces in man and machine', Neural Computation, vol. 18, no. 1, pp. 143-165. https://doi.org/10.1162/089976606774841611
Graf, Arnulf B A ; Wichmann, Felix A. ; Bulthoff, Heinrich ; Schölkopf, Bernhard. / Classification of faces in man and machine. In: Neural Computation. 2006 ; Vol. 18, No. 1. pp. 143-165.
@article{68f6df290d27499b89b48661b83665cd,
title = "Classification of faces in man and machine",
abstract = "We attempt to shed light on the algorithms humans use to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. First, we apply principal component analysis (PCA) on the pixel information of the face stimuli. We then obtain a data set composed of these PCA eigenvectors combined with the subjects' gender estimates of the corresponding stimuli. Second, we model the gender classification process on this data set using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning: the support vector machine (SVM), the relevance vector machine, the prototype classifier, and the K-means classifier. The classification behavior of humans and machines is then analyzed in three steps. First, the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects' internal decision boundary by studying the training errors of the machines. Second, we study the correlations between the rank-order of the subjects' responses to each stimulus - the gender estimate with its reaction time and confidence rating - and the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects' gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans. Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based on stimuli close to the boundary between the classes.",
author = "Graf, {Arnulf B A} and Wichmann, {Felix A.} and Heinrich Bulthoff and Bernhard Sch{\"o}lkopf",
year = "2006",
month = "1",
day = "1",
doi = "10.1162/089976606774841611",
language = "English",
volume = "18",
pages = "143--165",
journal = "Neural Computation",
issn = "0899-7667",
publisher = "MIT Press Journals",
number = "1",

}

TY - JOUR

T1 - Classification of faces in man and machine

AU - Graf, Arnulf B A

AU - Wichmann, Felix A.

AU - Bulthoff, Heinrich

AU - Schölkopf, Bernhard

PY - 2006/1/1

Y1 - 2006/1/1

N2 - We attempt to shed light on the algorithms humans use to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. First, we apply principal component analysis (PCA) on the pixel information of the face stimuli. We then obtain a data set composed of these PCA eigenvectors combined with the subjects' gender estimates of the corresponding stimuli. Second, we model the gender classification process on this data set using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning: the support vector machine (SVM), the relevance vector machine, the prototype classifier, and the K-means classifier. The classification behavior of humans and machines is then analyzed in three steps. First, the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects' internal decision boundary by studying the training errors of the machines. Second, we study the correlations between the rank-order of the subjects' responses to each stimulus - the gender estimate with its reaction time and confidence rating - and the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects' gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans. Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based on stimuli close to the boundary between the classes.

AB - We attempt to shed light on the algorithms humans use to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. First, we apply principal component analysis (PCA) on the pixel information of the face stimuli. We then obtain a data set composed of these PCA eigenvectors combined with the subjects' gender estimates of the corresponding stimuli. Second, we model the gender classification process on this data set using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning: the support vector machine (SVM), the relevance vector machine, the prototype classifier, and the K-means classifier. The classification behavior of humans and machines is then analyzed in three steps. First, the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects' internal decision boundary by studying the training errors of the machines. Second, we study the correlations between the rank-order of the subjects' responses to each stimulus - the gender estimate with its reaction time and confidence rating - and the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects' gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans. Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based on stimuli close to the boundary between the classes.

UR - http://www.scopus.com/inward/record.url?scp=33644875066&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33644875066&partnerID=8YFLogxK

U2 - 10.1162/089976606774841611

DO - 10.1162/089976606774841611

M3 - Article

C2 - 16354384

AN - SCOPUS:33644875066

VL - 18

SP - 143

EP - 165

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 1

ER -