Efficient model selection for probabilistic K nearest neighbour classification

Ji Won Yoon, Nial Friel

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Probabilistic K-nearest neighbour (PKNN) classification has been introduced to improve the performance of the original K-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, K. The contribution of this paper is to incorporate the uncertainty in K into the decision making, and consequently to provide improved classification with Bayesian model averaging. Indeed the problem of assessing the uncertainty in K can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, we develop a new functional approximation algorithm to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, the algorithms avoid cross validation by adopting Bayesian framework. The performance of the proposed approaches is evaluated on several real experimental datasets.

Original languageEnglish
Pages (from-to)1098-1108
Number of pages11
JournalNeurocomputing
Volume149
Issue numberPB
DOIs
Publication statusPublished - 2015 Feb 3

Fingerprint

Statistical Models
Uncertainty
Approximation algorithms
Learning systems
Decision Making
Decision making
Statistics

Keywords

  • Bayesian inference
  • K-free model order estimation
  • Model averaging

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Cognitive Neuroscience

Cite this

Efficient model selection for probabilistic K nearest neighbour classification. / Yoon, Ji Won; Friel, Nial.

In: Neurocomputing, Vol. 149, No. PB, 03.02.2015, p. 1098-1108.

Research output: Contribution to journalArticle

@article{f04efdfa0a3e434397741bc6d8f67261,
title = "Efficient model selection for probabilistic K nearest neighbour classification",
abstract = "Probabilistic K-nearest neighbour (PKNN) classification has been introduced to improve the performance of the original K-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, K. The contribution of this paper is to incorporate the uncertainty in K into the decision making, and consequently to provide improved classification with Bayesian model averaging. Indeed the problem of assessing the uncertainty in K can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, we develop a new functional approximation algorithm to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, the algorithms avoid cross validation by adopting Bayesian framework. The performance of the proposed approaches is evaluated on several real experimental datasets.",
keywords = "Bayesian inference, K-free model order estimation, Model averaging",
author = "Yoon, {Ji Won} and Nial Friel",
year = "2015",
month = "2",
day = "3",
doi = "10.1016/j.neucom.2014.07.023",
language = "English",
volume = "149",
pages = "1098--1108",
journal = "Neurocomputing",
issn = "0925-2312",
publisher = "Elsevier",
number = "PB",

}

TY - JOUR

T1 - Efficient model selection for probabilistic K nearest neighbour classification

AU - Yoon, Ji Won

AU - Friel, Nial

PY - 2015/2/3

Y1 - 2015/2/3

N2 - Probabilistic K-nearest neighbour (PKNN) classification has been introduced to improve the performance of the original K-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, K. The contribution of this paper is to incorporate the uncertainty in K into the decision making, and consequently to provide improved classification with Bayesian model averaging. Indeed the problem of assessing the uncertainty in K can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, we develop a new functional approximation algorithm to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, the algorithms avoid cross validation by adopting Bayesian framework. The performance of the proposed approaches is evaluated on several real experimental datasets.

AB - Probabilistic K-nearest neighbour (PKNN) classification has been introduced to improve the performance of the original K-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, K. The contribution of this paper is to incorporate the uncertainty in K into the decision making, and consequently to provide improved classification with Bayesian model averaging. Indeed the problem of assessing the uncertainty in K can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, we develop a new functional approximation algorithm to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, the algorithms avoid cross validation by adopting Bayesian framework. The performance of the proposed approaches is evaluated on several real experimental datasets.

KW - Bayesian inference

KW - K-free model order estimation

KW - Model averaging

UR - http://www.scopus.com/inward/record.url?scp=84922574118&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84922574118&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2014.07.023

DO - 10.1016/j.neucom.2014.07.023

M3 - Article

AN - SCOPUS:84922574118

VL - 149

SP - 1098

EP - 1108

JO - Neurocomputing

JF - Neurocomputing

SN - 0925-2312

IS - PB

ER -