Deep Boltzmann machines and the centering trick

Grégoire Montavon, Klaus Robert Müller

Research output: Chapter in Book/Report/Conference proceedingChapter

33 Citations (Scopus)

Abstract

Deep Boltzmann machines are in theory capable of learning efficient representations of seemingly complex data. Designing an algorithm that effectively learns the data representation can be subject to multiple difficulties. In this chapter, we present the "centering trick" that consists of rewriting the energy of the system as a function of centered states. The centering trick improves the conditioning of the underlying optimization problem and makes learning more stable, leading to models with better generative and discriminative properties.

Original languageEnglish
Title of host publicationNeural Networks
Subtitle of host publicationTricks of the Trade
EditorsGregoire Montavon, Klaus-Robert Muller, Genevieve B. Orr, Klaus-Robert Muller
Pages621-637
Number of pages17
DOIs
Publication statusPublished - 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7700 LECTURE NO
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • Deep Boltzmann machine
  • centering
  • optimization
  • reparameterization
  • representations
  • unsupervised learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Deep Boltzmann machines and the centering trick'. Together they form a unique fingerprint.

  • Cite this

    Montavon, G., & Müller, K. R. (2012). Deep Boltzmann machines and the centering trick. In G. Montavon, K-R. Muller, G. B. Orr, & K-R. Muller (Eds.), Neural Networks: Tricks of the Trade (pp. 621-637). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7700 LECTURE NO). https://doi.org/10.1007/978-3-642-35289-8-33