Locally linear reconstruction for instance-based learning

Pilsung Kang, Sungzoon Cho

Research output: Contribution to journalArticle

46 Citations (Scopus)

Abstract

Instance-based learning (IBL), so called memory-based reasoning (MBR), is a commonly used non-parametric learning algorithm. k-nearest neighbor (k-NN) learning is the most popular realization of IBL. Due to its usability and adaptability, k-NN has been successfully applied to a wide range of applications. However, in practice, one has to set important model parameters only empirically: the number of neighbors (k) and weights to those neighbors. In this paper, we propose structured ways to set these parameters, based on locally linear reconstruction (LLR). We then employed sequential minimal optimization (SMO) for solving quadratic programming step involved in LLR for classification to reduce the computational complexity. Experimental results from 11 classification and eight regression tasks were promising enough to merit further investigation: not only did LLR outperform the conventional weight allocation methods without much additional computational cost, but also LLR was found to be robust to the change of k.

Original languageEnglish
Pages (from-to)3507-3518
Number of pages12
JournalPattern Recognition
Volume41
Issue number11
DOIs
Publication statusPublished - 2008 Nov 1
Externally publishedYes

Fingerprint

Quadratic programming
Learning algorithms
Computational complexity
Data storage equipment
Costs

Keywords

  • Instance-based learning
  • k-nearest neighbor
  • Local reconstruction
  • Memory-based reasoning
  • Weight allocation

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Locally linear reconstruction for instance-based learning. / Kang, Pilsung; Cho, Sungzoon.

In: Pattern Recognition, Vol. 41, No. 11, 01.11.2008, p. 3507-3518.

Research output: Contribution to journalArticle

Kang, Pilsung ; Cho, Sungzoon. / Locally linear reconstruction for instance-based learning. In: Pattern Recognition. 2008 ; Vol. 41, No. 11. pp. 3507-3518.
@article{16165e7ace324ea8a02d05debf17da34,
title = "Locally linear reconstruction for instance-based learning",
abstract = "Instance-based learning (IBL), so called memory-based reasoning (MBR), is a commonly used non-parametric learning algorithm. k-nearest neighbor (k-NN) learning is the most popular realization of IBL. Due to its usability and adaptability, k-NN has been successfully applied to a wide range of applications. However, in practice, one has to set important model parameters only empirically: the number of neighbors (k) and weights to those neighbors. In this paper, we propose structured ways to set these parameters, based on locally linear reconstruction (LLR). We then employed sequential minimal optimization (SMO) for solving quadratic programming step involved in LLR for classification to reduce the computational complexity. Experimental results from 11 classification and eight regression tasks were promising enough to merit further investigation: not only did LLR outperform the conventional weight allocation methods without much additional computational cost, but also LLR was found to be robust to the change of k.",
keywords = "Instance-based learning, k-nearest neighbor, Local reconstruction, Memory-based reasoning, Weight allocation",
author = "Pilsung Kang and Sungzoon Cho",
year = "2008",
month = "11",
day = "1",
doi = "10.1016/j.patcog.2008.04.009",
language = "English",
volume = "41",
pages = "3507--3518",
journal = "Pattern Recognition",
issn = "0031-3203",
publisher = "Elsevier Limited",
number = "11",

}

TY - JOUR

T1 - Locally linear reconstruction for instance-based learning

AU - Kang, Pilsung

AU - Cho, Sungzoon

PY - 2008/11/1

Y1 - 2008/11/1

N2 - Instance-based learning (IBL), so called memory-based reasoning (MBR), is a commonly used non-parametric learning algorithm. k-nearest neighbor (k-NN) learning is the most popular realization of IBL. Due to its usability and adaptability, k-NN has been successfully applied to a wide range of applications. However, in practice, one has to set important model parameters only empirically: the number of neighbors (k) and weights to those neighbors. In this paper, we propose structured ways to set these parameters, based on locally linear reconstruction (LLR). We then employed sequential minimal optimization (SMO) for solving quadratic programming step involved in LLR for classification to reduce the computational complexity. Experimental results from 11 classification and eight regression tasks were promising enough to merit further investigation: not only did LLR outperform the conventional weight allocation methods without much additional computational cost, but also LLR was found to be robust to the change of k.

AB - Instance-based learning (IBL), so called memory-based reasoning (MBR), is a commonly used non-parametric learning algorithm. k-nearest neighbor (k-NN) learning is the most popular realization of IBL. Due to its usability and adaptability, k-NN has been successfully applied to a wide range of applications. However, in practice, one has to set important model parameters only empirically: the number of neighbors (k) and weights to those neighbors. In this paper, we propose structured ways to set these parameters, based on locally linear reconstruction (LLR). We then employed sequential minimal optimization (SMO) for solving quadratic programming step involved in LLR for classification to reduce the computational complexity. Experimental results from 11 classification and eight regression tasks were promising enough to merit further investigation: not only did LLR outperform the conventional weight allocation methods without much additional computational cost, but also LLR was found to be robust to the change of k.

KW - Instance-based learning

KW - k-nearest neighbor

KW - Local reconstruction

KW - Memory-based reasoning

KW - Weight allocation

UR - http://www.scopus.com/inward/record.url?scp=48149111512&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=48149111512&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2008.04.009

DO - 10.1016/j.patcog.2008.04.009

M3 - Article

AN - SCOPUS:48149111512

VL - 41

SP - 3507

EP - 3518

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

IS - 11

ER -