Abstract
Deep Boltzmann machines are in theory capable of learning efficient representations of seemingly complex data. Designing an algorithm that effectively learns the data representation can be subject to multiple difficulties. In this chapter, we present the "centering trick" that consists of rewriting the energy of the system as a function of centered states. The centering trick improves the conditioning of the underlying optimization problem and makes learning more stable, leading to models with better generative and discriminative properties.
| Original language | English |
|---|---|
| Title of host publication | Neural Networks |
| Subtitle of host publication | Tricks of the Trade |
| Editors | Gregoire Montavon, Klaus-Robert Muller, Genevieve B. Orr, Klaus-Robert Muller |
| Pages | 621-637 |
| Number of pages | 17 |
| DOIs | |
| Publication status | Published - 2012 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 7700 LECTURE NO |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Keywords
- Deep Boltzmann machine
- centering
- optimization
- reparameterization
- representations
- unsupervised learning
ASJC Scopus subject areas
- Theoretical Computer Science
- Computer Science(all)
Fingerprint Dive into the research topics of 'Deep Boltzmann machines and the centering trick'. Together they form a unique fingerprint.
Cite this
Montavon, G., & Müller, K. R. (2012). Deep Boltzmann machines and the centering trick. In G. Montavon, K-R. Muller, G. B. Orr, & K-R. Muller (Eds.), Neural Networks: Tricks of the Trade (pp. 621-637). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7700 LECTURE NO). https://doi.org/10.1007/978-3-642-35289-8-33