A model selection method based on bound of learning coefficient

Keisuke Yamazaki, Kenji Nagata, Sumio Watanabe, Klaus Muller

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

To decide the optimal size of learning machines is a central issue in the statistical learning theory, and that is why some theoretical criteria such as the BIG are developed. However, they cannot be applied to singular machines, and it is known that many practical learning machines e.g. mixture models, hidden Markov models, and Bayesian networks, are singular. Recently, we proposed the Singular Information Criterion (SingIC), which allows us to select the optimal size of singular machines. The SingIC is based on the analysis of the learning coefficient. So, the machines, to which the SingIC can be applied, are still limited. In this paper, we propose an extension of this criterion, which enables us to apply it to many singular machines, and evaluate the efficiency in Gaussian mixtures. The results offer an effective strategy to select the optimal size.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages371-380
Number of pages10
Volume4132 LNCS - II
Publication statusPublished - 2006 Oct 19
Externally publishedYes
Event16th International Conference on Artificial Neural Networks, ICANN 2006 - Athens, Greece
Duration: 2006 Sep 102006 Sep 14

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4132 LNCS - II
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other16th International Conference on Artificial Neural Networks, ICANN 2006
CountryGreece
CityAthens
Period06/9/1006/9/14

Fingerprint

Model Selection
Learning systems
Learning
Bayesian networks
Coefficient
Hidden Markov models
Information Criterion
Efficiency
Machine Learning
Statistical Learning Theory
Gaussian Mixture
Bayesian Networks
Mixture Model
Markov Model
Evaluate

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Yamazaki, K., Nagata, K., Watanabe, S., & Muller, K. (2006). A model selection method based on bound of learning coefficient. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 4132 LNCS - II, pp. 371-380). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4132 LNCS - II).

A model selection method based on bound of learning coefficient. / Yamazaki, Keisuke; Nagata, Kenji; Watanabe, Sumio; Muller, Klaus.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4132 LNCS - II 2006. p. 371-380 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4132 LNCS - II).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yamazaki, K, Nagata, K, Watanabe, S & Muller, K 2006, A model selection method based on bound of learning coefficient. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 4132 LNCS - II, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4132 LNCS - II, pp. 371-380, 16th International Conference on Artificial Neural Networks, ICANN 2006, Athens, Greece, 06/9/10.
Yamazaki K, Nagata K, Watanabe S, Muller K. A model selection method based on bound of learning coefficient. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4132 LNCS - II. 2006. p. 371-380. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Yamazaki, Keisuke ; Nagata, Kenji ; Watanabe, Sumio ; Muller, Klaus. / A model selection method based on bound of learning coefficient. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4132 LNCS - II 2006. pp. 371-380 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{dd6b2ea1ce1b45babbd32395ba643f28,
title = "A model selection method based on bound of learning coefficient",
abstract = "To decide the optimal size of learning machines is a central issue in the statistical learning theory, and that is why some theoretical criteria such as the BIG are developed. However, they cannot be applied to singular machines, and it is known that many practical learning machines e.g. mixture models, hidden Markov models, and Bayesian networks, are singular. Recently, we proposed the Singular Information Criterion (SingIC), which allows us to select the optimal size of singular machines. The SingIC is based on the analysis of the learning coefficient. So, the machines, to which the SingIC can be applied, are still limited. In this paper, we propose an extension of this criterion, which enables us to apply it to many singular machines, and evaluate the efficiency in Gaussian mixtures. The results offer an effective strategy to select the optimal size.",
author = "Keisuke Yamazaki and Kenji Nagata and Sumio Watanabe and Klaus Muller",
year = "2006",
month = "10",
day = "19",
language = "English",
isbn = "3540388710",
volume = "4132 LNCS - II",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "371--380",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - A model selection method based on bound of learning coefficient

AU - Yamazaki, Keisuke

AU - Nagata, Kenji

AU - Watanabe, Sumio

AU - Muller, Klaus

PY - 2006/10/19

Y1 - 2006/10/19

N2 - To decide the optimal size of learning machines is a central issue in the statistical learning theory, and that is why some theoretical criteria such as the BIG are developed. However, they cannot be applied to singular machines, and it is known that many practical learning machines e.g. mixture models, hidden Markov models, and Bayesian networks, are singular. Recently, we proposed the Singular Information Criterion (SingIC), which allows us to select the optimal size of singular machines. The SingIC is based on the analysis of the learning coefficient. So, the machines, to which the SingIC can be applied, are still limited. In this paper, we propose an extension of this criterion, which enables us to apply it to many singular machines, and evaluate the efficiency in Gaussian mixtures. The results offer an effective strategy to select the optimal size.

AB - To decide the optimal size of learning machines is a central issue in the statistical learning theory, and that is why some theoretical criteria such as the BIG are developed. However, they cannot be applied to singular machines, and it is known that many practical learning machines e.g. mixture models, hidden Markov models, and Bayesian networks, are singular. Recently, we proposed the Singular Information Criterion (SingIC), which allows us to select the optimal size of singular machines. The SingIC is based on the analysis of the learning coefficient. So, the machines, to which the SingIC can be applied, are still limited. In this paper, we propose an extension of this criterion, which enables us to apply it to many singular machines, and evaluate the efficiency in Gaussian mixtures. The results offer an effective strategy to select the optimal size.

UR - http://www.scopus.com/inward/record.url?scp=33749838218&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33749838218&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:33749838218

SN - 3540388710

SN - 9783540388715

VL - 4132 LNCS - II

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 371

EP - 380

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -