On relevant dimensions in kernel feature spaces

Mikio L. Braun, Joachim M. Buhmann, Klaus Muller

Research output: Contribution to journalArticle

78 Citations (Scopus)

Abstract

We show that the relevant information of a supervised learning problem is contained up to negligible error in a finite number of leading kernel PCA components if the kernel matches the underlying learning problem in the sense that it can asymptotically represent the function to be learned and is sufficiently smooth. Thus, kernels do not only transform data sets such that good generalization can be achieved using only linear discriminant functions, but this transformation is also performed in a manner which makes economical use of feature space dimensions. In the best case, kernels provide efficient implicit representations of the data for supervised learning problems. Practically, we propose an algorithm which enables us to recover the number of leading kernel PCA components relevant for good classification. Our algorithm can therefore be applied (1) to analyze the interplay of data set and kernel in a geometric fashion, (2) to aid in model selection, and (3) to denoise in feature space in order to yield better classification results.

Original languageEnglish
Pages (from-to)1875-1908
Number of pages34
JournalJournal of Machine Learning Research
Volume9
Publication statusPublished - 2008 Aug 1
Externally publishedYes

Fingerprint

Supervised learning
Feature Space
Kernel PCA
kernel
Supervised Learning
Linear Discriminant Function
Model Selection
Transform

Keywords

  • Dimension reduction
  • Effective dimensionality
  • Feature space
  • Kernel methods

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Braun, M. L., Buhmann, J. M., & Muller, K. (2008). On relevant dimensions in kernel feature spaces. Journal of Machine Learning Research, 9, 1875-1908.

On relevant dimensions in kernel feature spaces. / Braun, Mikio L.; Buhmann, Joachim M.; Muller, Klaus.

In: Journal of Machine Learning Research, Vol. 9, 01.08.2008, p. 1875-1908.

Research output: Contribution to journalArticle

Braun, ML, Buhmann, JM & Muller, K 2008, 'On relevant dimensions in kernel feature spaces', Journal of Machine Learning Research, vol. 9, pp. 1875-1908.
Braun ML, Buhmann JM, Muller K. On relevant dimensions in kernel feature spaces. Journal of Machine Learning Research. 2008 Aug 1;9:1875-1908.
Braun, Mikio L. ; Buhmann, Joachim M. ; Muller, Klaus. / On relevant dimensions in kernel feature spaces. In: Journal of Machine Learning Research. 2008 ; Vol. 9. pp. 1875-1908.
@article{120a2d5f3a0342b5b5d78f62b8848c07,
title = "On relevant dimensions in kernel feature spaces",
abstract = "We show that the relevant information of a supervised learning problem is contained up to negligible error in a finite number of leading kernel PCA components if the kernel matches the underlying learning problem in the sense that it can asymptotically represent the function to be learned and is sufficiently smooth. Thus, kernels do not only transform data sets such that good generalization can be achieved using only linear discriminant functions, but this transformation is also performed in a manner which makes economical use of feature space dimensions. In the best case, kernels provide efficient implicit representations of the data for supervised learning problems. Practically, we propose an algorithm which enables us to recover the number of leading kernel PCA components relevant for good classification. Our algorithm can therefore be applied (1) to analyze the interplay of data set and kernel in a geometric fashion, (2) to aid in model selection, and (3) to denoise in feature space in order to yield better classification results.",
keywords = "Dimension reduction, Effective dimensionality, Feature space, Kernel methods",
author = "Braun, {Mikio L.} and Buhmann, {Joachim M.} and Klaus Muller",
year = "2008",
month = "8",
day = "1",
language = "English",
volume = "9",
pages = "1875--1908",
journal = "Journal of Machine Learning Research",
issn = "1532-4435",
publisher = "Microtome Publishing",

}

TY - JOUR

T1 - On relevant dimensions in kernel feature spaces

AU - Braun, Mikio L.

AU - Buhmann, Joachim M.

AU - Muller, Klaus

PY - 2008/8/1

Y1 - 2008/8/1

N2 - We show that the relevant information of a supervised learning problem is contained up to negligible error in a finite number of leading kernel PCA components if the kernel matches the underlying learning problem in the sense that it can asymptotically represent the function to be learned and is sufficiently smooth. Thus, kernels do not only transform data sets such that good generalization can be achieved using only linear discriminant functions, but this transformation is also performed in a manner which makes economical use of feature space dimensions. In the best case, kernels provide efficient implicit representations of the data for supervised learning problems. Practically, we propose an algorithm which enables us to recover the number of leading kernel PCA components relevant for good classification. Our algorithm can therefore be applied (1) to analyze the interplay of data set and kernel in a geometric fashion, (2) to aid in model selection, and (3) to denoise in feature space in order to yield better classification results.

AB - We show that the relevant information of a supervised learning problem is contained up to negligible error in a finite number of leading kernel PCA components if the kernel matches the underlying learning problem in the sense that it can asymptotically represent the function to be learned and is sufficiently smooth. Thus, kernels do not only transform data sets such that good generalization can be achieved using only linear discriminant functions, but this transformation is also performed in a manner which makes economical use of feature space dimensions. In the best case, kernels provide efficient implicit representations of the data for supervised learning problems. Practically, we propose an algorithm which enables us to recover the number of leading kernel PCA components relevant for good classification. Our algorithm can therefore be applied (1) to analyze the interplay of data set and kernel in a geometric fashion, (2) to aid in model selection, and (3) to denoise in feature space in order to yield better classification results.

KW - Dimension reduction

KW - Effective dimensionality

KW - Feature space

KW - Kernel methods

UR - http://www.scopus.com/inward/record.url?scp=50949127418&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=50949127418&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:50949127418

VL - 9

SP - 1875

EP - 1908

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

SN - 1532-4435

ER -