Online learning of sparse pseudo-input Gaussian process

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we propose a novel method of online learning of sparse pseudo-data, representative of the whole training data, for Gaussian Process (GP) regressions. We call the proposed method Incremental Sparse Pseudo-input Gaussian Process (ISPGP) regression. The proposed ISPGP algorithm allows for training from either a huge amount of training data by scanning through it only once or an online incremental training dataset. Thanks to the nature of the incremental learning algorithm, the proposed ISPGP algorithm can theoretically work with infinite data to which the conventional GP or SPGP algorithm is not applicable. From our experimental results on the KIN40K dataset, we can see that the proposed ISPGP algorithm is comparable to the conventional GP algorithm using the same number of training data. Although the proposed ISPGP algorithm performs slightly worse than Snelson and Ghahramani's SPGP algorithm, the level of performance degradation is acceptable.

Original languageEnglish
Title of host publicationConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
Pages1357-1360
Number of pages4
DOIs
Publication statusPublished - 2012 Dec 1
Event2012 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2012 - Seoul, Korea, Republic of
Duration: 2012 Oct 142012 Oct 17

Other

Other2012 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2012
CountryKorea, Republic of
CitySeoul
Period12/10/1412/10/17

Fingerprint

Learning algorithms
Scanning
Degradation

Keywords

  • Gaussian process regression
  • incremental learning
  • pseudo-input

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Human-Computer Interaction

Cite this

Suk, H-I., Wang, Y., & Lee, S. W. (2012). Online learning of sparse pseudo-input Gaussian process. In Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics (pp. 1357-1360). [6377922] https://doi.org/10.1109/ICSMC.2012.6377922

Online learning of sparse pseudo-input Gaussian process. / Suk, Heung-Il; Wang, Yuzhuo; Lee, Seong Whan.

Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics. 2012. p. 1357-1360 6377922.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Suk, H-I, Wang, Y & Lee, SW 2012, Online learning of sparse pseudo-input Gaussian process. in Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics., 6377922, pp. 1357-1360, 2012 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2012, Seoul, Korea, Republic of, 12/10/14. https://doi.org/10.1109/ICSMC.2012.6377922
Suk H-I, Wang Y, Lee SW. Online learning of sparse pseudo-input Gaussian process. In Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics. 2012. p. 1357-1360. 6377922 https://doi.org/10.1109/ICSMC.2012.6377922
Suk, Heung-Il ; Wang, Yuzhuo ; Lee, Seong Whan. / Online learning of sparse pseudo-input Gaussian process. Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics. 2012. pp. 1357-1360
@inproceedings{601122664a254277a79b9ae9d0e61c53,
title = "Online learning of sparse pseudo-input Gaussian process",
abstract = "In this paper, we propose a novel method of online learning of sparse pseudo-data, representative of the whole training data, for Gaussian Process (GP) regressions. We call the proposed method Incremental Sparse Pseudo-input Gaussian Process (ISPGP) regression. The proposed ISPGP algorithm allows for training from either a huge amount of training data by scanning through it only once or an online incremental training dataset. Thanks to the nature of the incremental learning algorithm, the proposed ISPGP algorithm can theoretically work with infinite data to which the conventional GP or SPGP algorithm is not applicable. From our experimental results on the KIN40K dataset, we can see that the proposed ISPGP algorithm is comparable to the conventional GP algorithm using the same number of training data. Although the proposed ISPGP algorithm performs slightly worse than Snelson and Ghahramani's SPGP algorithm, the level of performance degradation is acceptable.",
keywords = "Gaussian process regression, incremental learning, pseudo-input",
author = "Heung-Il Suk and Yuzhuo Wang and Lee, {Seong Whan}",
year = "2012",
month = "12",
day = "1",
doi = "10.1109/ICSMC.2012.6377922",
language = "English",
isbn = "9781467317146",
pages = "1357--1360",
booktitle = "Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics",

}

TY - GEN

T1 - Online learning of sparse pseudo-input Gaussian process

AU - Suk, Heung-Il

AU - Wang, Yuzhuo

AU - Lee, Seong Whan

PY - 2012/12/1

Y1 - 2012/12/1

N2 - In this paper, we propose a novel method of online learning of sparse pseudo-data, representative of the whole training data, for Gaussian Process (GP) regressions. We call the proposed method Incremental Sparse Pseudo-input Gaussian Process (ISPGP) regression. The proposed ISPGP algorithm allows for training from either a huge amount of training data by scanning through it only once or an online incremental training dataset. Thanks to the nature of the incremental learning algorithm, the proposed ISPGP algorithm can theoretically work with infinite data to which the conventional GP or SPGP algorithm is not applicable. From our experimental results on the KIN40K dataset, we can see that the proposed ISPGP algorithm is comparable to the conventional GP algorithm using the same number of training data. Although the proposed ISPGP algorithm performs slightly worse than Snelson and Ghahramani's SPGP algorithm, the level of performance degradation is acceptable.

AB - In this paper, we propose a novel method of online learning of sparse pseudo-data, representative of the whole training data, for Gaussian Process (GP) regressions. We call the proposed method Incremental Sparse Pseudo-input Gaussian Process (ISPGP) regression. The proposed ISPGP algorithm allows for training from either a huge amount of training data by scanning through it only once or an online incremental training dataset. Thanks to the nature of the incremental learning algorithm, the proposed ISPGP algorithm can theoretically work with infinite data to which the conventional GP or SPGP algorithm is not applicable. From our experimental results on the KIN40K dataset, we can see that the proposed ISPGP algorithm is comparable to the conventional GP algorithm using the same number of training data. Although the proposed ISPGP algorithm performs slightly worse than Snelson and Ghahramani's SPGP algorithm, the level of performance degradation is acceptable.

KW - Gaussian process regression

KW - incremental learning

KW - pseudo-input

UR - http://www.scopus.com/inward/record.url?scp=84872419813&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84872419813&partnerID=8YFLogxK

U2 - 10.1109/ICSMC.2012.6377922

DO - 10.1109/ICSMC.2012.6377922

M3 - Conference contribution

SN - 9781467317146

SP - 1357

EP - 1360

BT - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

ER -