Deep Boltzmann machines and the centering trick

Grégoire Montavon, Klaus Robert Müller

Research output: Chapter in Book/Report/Conference proceedingChapter

36 Citations (Scopus)

Abstract

Deep Boltzmann machines are in theory capable of learning efficient representations of seemingly complex data. Designing an algorithm that effectively learns the data representation can be subject to multiple difficulties. In this chapter, we present the "centering trick" that consists of rewriting the energy of the system as a function of centered states. The centering trick improves the conditioning of the underlying optimization problem and makes learning more stable, leading to models with better generative and discriminative properties.

Original languageEnglish
Title of host publicationNeural Networks
Subtitle of host publicationTricks of the Trade
PublisherSpringer Verlag
Pages621-637
Number of pages17
ISBN (Print)9783642352881
DOIs
Publication statusPublished - 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7700 LECTURE NO
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • Deep Boltzmann machine
  • centering
  • optimization
  • reparameterization
  • representations
  • unsupervised learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Deep Boltzmann machines and the centering trick'. Together they form a unique fingerprint.

Cite this