An empirical study on the properties of random bases for kernel methods

Maximilian Alber, Pieter Jan Kindermans, Kristof T. Schütt, Klaus Muller, Fei Sha

Research output: Contribution to journalConference article

2 Citations (Scopus)

Abstract

Kernel machines as well as neural networks possess universal function approximation properties. Nevertheless in practice their ways of choosing the appropriate function class differ. Specifically neural networks learn a representation by adapting their basis functions to the data and the task at hand, while kernel methods typically use a basis that is not adapted during training. In this work, we contrast random features of approximated kernel machines with learned features of neural networks. Our analysis reveals how these random and adaptive basis functions affect the quality of learning. Furthermore, we present basis adaptation schemes that allow for a more compact representation, while retaining the generalization properties of kernel machines.

Original languageEnglish
Pages (from-to)2764-2775
Number of pages12
JournalAdvances in Neural Information Processing Systems
Volume2017-December
Publication statusPublished - 2017 Jan 1
Event31st Annual Conference on Neural Information Processing Systems, NIPS 2017 - Long Beach, United States
Duration: 2017 Dec 42017 Dec 9

Fingerprint

Neural networks

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

Alber, M., Kindermans, P. J., Schütt, K. T., Muller, K., & Sha, F. (2017). An empirical study on the properties of random bases for kernel methods. Advances in Neural Information Processing Systems, 2017-December, 2764-2775.

An empirical study on the properties of random bases for kernel methods. / Alber, Maximilian; Kindermans, Pieter Jan; Schütt, Kristof T.; Muller, Klaus; Sha, Fei.

In: Advances in Neural Information Processing Systems, Vol. 2017-December, 01.01.2017, p. 2764-2775.

Research output: Contribution to journalConference article

Alber, M, Kindermans, PJ, Schütt, KT, Muller, K & Sha, F 2017, 'An empirical study on the properties of random bases for kernel methods', Advances in Neural Information Processing Systems, vol. 2017-December, pp. 2764-2775.
Alber M, Kindermans PJ, Schütt KT, Muller K, Sha F. An empirical study on the properties of random bases for kernel methods. Advances in Neural Information Processing Systems. 2017 Jan 1;2017-December:2764-2775.
Alber, Maximilian ; Kindermans, Pieter Jan ; Schütt, Kristof T. ; Muller, Klaus ; Sha, Fei. / An empirical study on the properties of random bases for kernel methods. In: Advances in Neural Information Processing Systems. 2017 ; Vol. 2017-December. pp. 2764-2775.
@article{d9950dbc43aa4aee8c453cc83a347991,
title = "An empirical study on the properties of random bases for kernel methods",
abstract = "Kernel machines as well as neural networks possess universal function approximation properties. Nevertheless in practice their ways of choosing the appropriate function class differ. Specifically neural networks learn a representation by adapting their basis functions to the data and the task at hand, while kernel methods typically use a basis that is not adapted during training. In this work, we contrast random features of approximated kernel machines with learned features of neural networks. Our analysis reveals how these random and adaptive basis functions affect the quality of learning. Furthermore, we present basis adaptation schemes that allow for a more compact representation, while retaining the generalization properties of kernel machines.",
author = "Maximilian Alber and Kindermans, {Pieter Jan} and Sch{\"u}tt, {Kristof T.} and Klaus Muller and Fei Sha",
year = "2017",
month = "1",
day = "1",
language = "English",
volume = "2017-December",
pages = "2764--2775",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",

}

TY - JOUR

T1 - An empirical study on the properties of random bases for kernel methods

AU - Alber, Maximilian

AU - Kindermans, Pieter Jan

AU - Schütt, Kristof T.

AU - Muller, Klaus

AU - Sha, Fei

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Kernel machines as well as neural networks possess universal function approximation properties. Nevertheless in practice their ways of choosing the appropriate function class differ. Specifically neural networks learn a representation by adapting their basis functions to the data and the task at hand, while kernel methods typically use a basis that is not adapted during training. In this work, we contrast random features of approximated kernel machines with learned features of neural networks. Our analysis reveals how these random and adaptive basis functions affect the quality of learning. Furthermore, we present basis adaptation schemes that allow for a more compact representation, while retaining the generalization properties of kernel machines.

AB - Kernel machines as well as neural networks possess universal function approximation properties. Nevertheless in practice their ways of choosing the appropriate function class differ. Specifically neural networks learn a representation by adapting their basis functions to the data and the task at hand, while kernel methods typically use a basis that is not adapted during training. In this work, we contrast random features of approximated kernel machines with learned features of neural networks. Our analysis reveals how these random and adaptive basis functions affect the quality of learning. Furthermore, we present basis adaptation schemes that allow for a more compact representation, while retaining the generalization properties of kernel machines.

UR - http://www.scopus.com/inward/record.url?scp=85047001988&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047001988&partnerID=8YFLogxK

M3 - Conference article

VL - 2017-December

SP - 2764

EP - 2775

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -