Asymptotic Properties of the Fisher Kernel

Koji Tsuda, Shotaro Akaho, Motoaki Kawanabe, Klaus Muller

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

This letter analyzes the Fisher kernel from a statistical point of view. The Fisher kernel is a particularly interesting method for constructing a model of the posterior probability that makes intelligent use of unlabeled data (i.e., of the underlying data density). It is important to analyze and ultimately understand the statistical properties of the Fisher kernel. To this end, we first establish sufficient conditions that the constructed posterior model is realizable (i.e., it contains the true distribution). Realizability immediately leads to consistency results. Subsequently, we focus on an asymptotic analysis of the generalization error, which elucidates the learning curves of the Fisher kernel and how unlabeled data contribute to learning. We also point out that the squared or log loss is theoretically preferable - because both yield consistent estimators - to other losses such as the exponential loss, when a linear classifier is used together with the Fisher kernel. Therefore, this letter underlines that the Fisher kernel should be viewed not as a heuristics but as a powerful statistical tool with well-controlled statistical properties.

Original languageEnglish
Pages (from-to)115-137
Number of pages23
JournalNeural Computation
Volume16
Issue number1
DOIs
Publication statusPublished - 2004 Jan 1
Externally publishedYes

Fingerprint

Learning Curve
Learning
Asymptotic analysis
Classifiers
Generalization (Psychology)
Heuristics
Kernel

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Artificial Intelligence
  • Neuroscience(all)

Cite this

Tsuda, K., Akaho, S., Kawanabe, M., & Muller, K. (2004). Asymptotic Properties of the Fisher Kernel. Neural Computation, 16(1), 115-137. https://doi.org/10.1162/08997660460734029

Asymptotic Properties of the Fisher Kernel. / Tsuda, Koji; Akaho, Shotaro; Kawanabe, Motoaki; Muller, Klaus.

In: Neural Computation, Vol. 16, No. 1, 01.01.2004, p. 115-137.

Research output: Contribution to journalArticle

Tsuda, K, Akaho, S, Kawanabe, M & Muller, K 2004, 'Asymptotic Properties of the Fisher Kernel', Neural Computation, vol. 16, no. 1, pp. 115-137. https://doi.org/10.1162/08997660460734029
Tsuda K, Akaho S, Kawanabe M, Muller K. Asymptotic Properties of the Fisher Kernel. Neural Computation. 2004 Jan 1;16(1):115-137. https://doi.org/10.1162/08997660460734029
Tsuda, Koji ; Akaho, Shotaro ; Kawanabe, Motoaki ; Muller, Klaus. / Asymptotic Properties of the Fisher Kernel. In: Neural Computation. 2004 ; Vol. 16, No. 1. pp. 115-137.
@article{951d755def4d49859b3694bd20386116,
title = "Asymptotic Properties of the Fisher Kernel",
abstract = "This letter analyzes the Fisher kernel from a statistical point of view. The Fisher kernel is a particularly interesting method for constructing a model of the posterior probability that makes intelligent use of unlabeled data (i.e., of the underlying data density). It is important to analyze and ultimately understand the statistical properties of the Fisher kernel. To this end, we first establish sufficient conditions that the constructed posterior model is realizable (i.e., it contains the true distribution). Realizability immediately leads to consistency results. Subsequently, we focus on an asymptotic analysis of the generalization error, which elucidates the learning curves of the Fisher kernel and how unlabeled data contribute to learning. We also point out that the squared or log loss is theoretically preferable - because both yield consistent estimators - to other losses such as the exponential loss, when a linear classifier is used together with the Fisher kernel. Therefore, this letter underlines that the Fisher kernel should be viewed not as a heuristics but as a powerful statistical tool with well-controlled statistical properties.",
author = "Koji Tsuda and Shotaro Akaho and Motoaki Kawanabe and Klaus Muller",
year = "2004",
month = "1",
day = "1",
doi = "10.1162/08997660460734029",
language = "English",
volume = "16",
pages = "115--137",
journal = "Neural Computation",
issn = "0899-7667",
publisher = "MIT Press Journals",
number = "1",

}

TY - JOUR

T1 - Asymptotic Properties of the Fisher Kernel

AU - Tsuda, Koji

AU - Akaho, Shotaro

AU - Kawanabe, Motoaki

AU - Muller, Klaus

PY - 2004/1/1

Y1 - 2004/1/1

N2 - This letter analyzes the Fisher kernel from a statistical point of view. The Fisher kernel is a particularly interesting method for constructing a model of the posterior probability that makes intelligent use of unlabeled data (i.e., of the underlying data density). It is important to analyze and ultimately understand the statistical properties of the Fisher kernel. To this end, we first establish sufficient conditions that the constructed posterior model is realizable (i.e., it contains the true distribution). Realizability immediately leads to consistency results. Subsequently, we focus on an asymptotic analysis of the generalization error, which elucidates the learning curves of the Fisher kernel and how unlabeled data contribute to learning. We also point out that the squared or log loss is theoretically preferable - because both yield consistent estimators - to other losses such as the exponential loss, when a linear classifier is used together with the Fisher kernel. Therefore, this letter underlines that the Fisher kernel should be viewed not as a heuristics but as a powerful statistical tool with well-controlled statistical properties.

AB - This letter analyzes the Fisher kernel from a statistical point of view. The Fisher kernel is a particularly interesting method for constructing a model of the posterior probability that makes intelligent use of unlabeled data (i.e., of the underlying data density). It is important to analyze and ultimately understand the statistical properties of the Fisher kernel. To this end, we first establish sufficient conditions that the constructed posterior model is realizable (i.e., it contains the true distribution). Realizability immediately leads to consistency results. Subsequently, we focus on an asymptotic analysis of the generalization error, which elucidates the learning curves of the Fisher kernel and how unlabeled data contribute to learning. We also point out that the squared or log loss is theoretically preferable - because both yield consistent estimators - to other losses such as the exponential loss, when a linear classifier is used together with the Fisher kernel. Therefore, this letter underlines that the Fisher kernel should be viewed not as a heuristics but as a powerful statistical tool with well-controlled statistical properties.

UR - http://www.scopus.com/inward/record.url?scp=0346895562&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0346895562&partnerID=8YFLogxK

U2 - 10.1162/08997660460734029

DO - 10.1162/08997660460734029

M3 - Article

C2 - 15006026

AN - SCOPUS:0346895562

VL - 16

SP - 115

EP - 137

JO - Neural Computation

JF - Neural Computation

SN - 0899-7667

IS - 1

ER -