Nonlinear gradient denoising: Finding accurate extrema from inaccurate functional derivatives

John C. Snyder, Matthias Rupp, Klaus Muller, Kieron Burke

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

A method for nonlinear optimization with machine learning (ML) models, called nonlinear gradient denoising (NLGD), is developed, and applied with ML approximations to the kinetic energy density functional in an orbital-free density functional theory. Due to systematically inaccurate gradients of ML models, in particular when the data is very high-dimensional, the optimization must be constrained to the data manifold. We use nonlinear kernel principal component analysis (PCA) to locally reconstruct the manifold, enabling a projected gradient descent along it. A thorough analysis of the method is given via a simple model, designed to clarify the concepts presented. Additionally, NLGD is compared with the local PCA method used in previous work. Our method is shown to be superior in cases when the data manifold is highly nonlinear and high dimensional. Further applications of the method in both density functional theory and ML are discussed.

Original languageEnglish
Pages (from-to)1102-1114
Number of pages13
JournalInternational Journal of Quantum Chemistry
Volume115
Issue number16
DOIs
Publication statusPublished - 2015 Aug 1

Fingerprint

machine learning
range (extremes)
Learning systems
Derivatives
gradients
principal components analysis
Principal component analysis
Density functional theory
density functional theory
optimization
descent
Kinetic energy
flux density
kinetic energy
orbitals
approximation

Keywords

  • density functional theory
  • machine learning
  • nonlinear gradient denoising
  • orbital-free density functional theory

ASJC Scopus subject areas

  • Condensed Matter Physics
  • Atomic and Molecular Physics, and Optics
  • Physical and Theoretical Chemistry

Cite this

Nonlinear gradient denoising : Finding accurate extrema from inaccurate functional derivatives. / Snyder, John C.; Rupp, Matthias; Muller, Klaus; Burke, Kieron.

In: International Journal of Quantum Chemistry, Vol. 115, No. 16, 01.08.2015, p. 1102-1114.

Research output: Contribution to journalArticle

Snyder, John C. ; Rupp, Matthias ; Muller, Klaus ; Burke, Kieron. / Nonlinear gradient denoising : Finding accurate extrema from inaccurate functional derivatives. In: International Journal of Quantum Chemistry. 2015 ; Vol. 115, No. 16. pp. 1102-1114.
@article{d97f4fc243d44a648793ba52e2fa356e,
title = "Nonlinear gradient denoising: Finding accurate extrema from inaccurate functional derivatives",
abstract = "A method for nonlinear optimization with machine learning (ML) models, called nonlinear gradient denoising (NLGD), is developed, and applied with ML approximations to the kinetic energy density functional in an orbital-free density functional theory. Due to systematically inaccurate gradients of ML models, in particular when the data is very high-dimensional, the optimization must be constrained to the data manifold. We use nonlinear kernel principal component analysis (PCA) to locally reconstruct the manifold, enabling a projected gradient descent along it. A thorough analysis of the method is given via a simple model, designed to clarify the concepts presented. Additionally, NLGD is compared with the local PCA method used in previous work. Our method is shown to be superior in cases when the data manifold is highly nonlinear and high dimensional. Further applications of the method in both density functional theory and ML are discussed.",
keywords = "density functional theory, machine learning, nonlinear gradient denoising, orbital-free density functional theory",
author = "Snyder, {John C.} and Matthias Rupp and Klaus Muller and Kieron Burke",
year = "2015",
month = "8",
day = "1",
doi = "10.1002/qua.24937",
language = "English",
volume = "115",
pages = "1102--1114",
journal = "International Journal of Quantum Chemistry",
issn = "0020-7608",
publisher = "John Wiley and Sons Inc.",
number = "16",

}

TY - JOUR

T1 - Nonlinear gradient denoising

T2 - Finding accurate extrema from inaccurate functional derivatives

AU - Snyder, John C.

AU - Rupp, Matthias

AU - Muller, Klaus

AU - Burke, Kieron

PY - 2015/8/1

Y1 - 2015/8/1

N2 - A method for nonlinear optimization with machine learning (ML) models, called nonlinear gradient denoising (NLGD), is developed, and applied with ML approximations to the kinetic energy density functional in an orbital-free density functional theory. Due to systematically inaccurate gradients of ML models, in particular when the data is very high-dimensional, the optimization must be constrained to the data manifold. We use nonlinear kernel principal component analysis (PCA) to locally reconstruct the manifold, enabling a projected gradient descent along it. A thorough analysis of the method is given via a simple model, designed to clarify the concepts presented. Additionally, NLGD is compared with the local PCA method used in previous work. Our method is shown to be superior in cases when the data manifold is highly nonlinear and high dimensional. Further applications of the method in both density functional theory and ML are discussed.

AB - A method for nonlinear optimization with machine learning (ML) models, called nonlinear gradient denoising (NLGD), is developed, and applied with ML approximations to the kinetic energy density functional in an orbital-free density functional theory. Due to systematically inaccurate gradients of ML models, in particular when the data is very high-dimensional, the optimization must be constrained to the data manifold. We use nonlinear kernel principal component analysis (PCA) to locally reconstruct the manifold, enabling a projected gradient descent along it. A thorough analysis of the method is given via a simple model, designed to clarify the concepts presented. Additionally, NLGD is compared with the local PCA method used in previous work. Our method is shown to be superior in cases when the data manifold is highly nonlinear and high dimensional. Further applications of the method in both density functional theory and ML are discussed.

KW - density functional theory

KW - machine learning

KW - nonlinear gradient denoising

KW - orbital-free density functional theory

UR - http://www.scopus.com/inward/record.url?scp=84936751541&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84936751541&partnerID=8YFLogxK

U2 - 10.1002/qua.24937

DO - 10.1002/qua.24937

M3 - Article

AN - SCOPUS:84936751541

VL - 115

SP - 1102

EP - 1114

JO - International Journal of Quantum Chemistry

JF - International Journal of Quantum Chemistry

SN - 0020-7608

IS - 16

ER -