An empirical study on the properties of random bases for kernel methods

Maximilian Alber, Pieter Jan Kindermans, Kristof T. Schütt, Klaus Muller, Fei Sha

Research output: Contribution to journalConference article

3 Citations (Scopus)

Abstract

Kernel machines as well as neural networks possess universal function approximation properties. Nevertheless in practice their ways of choosing the appropriate function class differ. Specifically neural networks learn a representation by adapting their basis functions to the data and the task at hand, while kernel methods typically use a basis that is not adapted during training. In this work, we contrast random features of approximated kernel machines with learned features of neural networks. Our analysis reveals how these random and adaptive basis functions affect the quality of learning. Furthermore, we present basis adaptation schemes that allow for a more compact representation, while retaining the generalization properties of kernel machines.

Original languageEnglish
Pages (from-to)2764-2775
Number of pages12
JournalAdvances in Neural Information Processing Systems
Volume2017-December
Publication statusPublished - 2017 Jan 1
Event31st Annual Conference on Neural Information Processing Systems, NIPS 2017 - Long Beach, United States
Duration: 2017 Dec 42017 Dec 9

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint Dive into the research topics of 'An empirical study on the properties of random bases for kernel methods'. Together they form a unique fingerprint.

  • Cite this

    Alber, M., Kindermans, P. J., Schütt, K. T., Muller, K., & Sha, F. (2017). An empirical study on the properties of random bases for kernel methods. Advances in Neural Information Processing Systems, 2017-December, 2764-2775.