FBP: A frontier-based tree-pruning algorithm

Xiaoming Huo, Seoung Bum Kim, Kwok Leung Tsui, Shuchun Wang

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

A frontier-based tree-pruning algorithm (FBP) is proposed. The new method has an order of computational complexity comparable to cost-complexity pruning (CCP). Regarding tree pruning, it provides a full spectrum of information: namely, (1) given the value of the penalization parameter λ, it gives the decision tree specified by the complexity-penalization approach; (2) given the size of a decision tree, it provides the range of the penalization parameter λ, within which the complexity-penalization approach renders this tree size; (3) it finds the tree sizes that are inadmissible - no matter what the value of the penalty parameter is, the resulting tree based on a complexity-penalization framework will never have these sizes. Simulations on real data sets reveal a "surprise:" in the complexity-penalization approach, most of the tree sizes are inadmissible. FBP facilitates a more faithful implementation of cross validation (CV), which is favored by simulations. Using FBP, a stability analysis of CV is proposed.

Original languageEnglish
Pages (from-to)494-505
Number of pages12
JournalINFORMS Journal on Computing
Volume18
Issue number4
DOIs
Publication statusPublished - 2006 Sep 1
Externally publishedYes

Fingerprint

Trees (mathematics)
Decision trees
Computational complexity
Costs
Pruning

Keywords

  • Classification
  • Data mining
  • Decision trees
  • Tree pruning

ASJC Scopus subject areas

  • Computer Science Applications
  • Computational Theory and Mathematics
  • Management Science and Operations Research

Cite this

FBP : A frontier-based tree-pruning algorithm. / Huo, Xiaoming; Kim, Seoung Bum; Tsui, Kwok Leung; Wang, Shuchun.

In: INFORMS Journal on Computing, Vol. 18, No. 4, 01.09.2006, p. 494-505.

Research output: Contribution to journalArticle

Huo, Xiaoming ; Kim, Seoung Bum ; Tsui, Kwok Leung ; Wang, Shuchun. / FBP : A frontier-based tree-pruning algorithm. In: INFORMS Journal on Computing. 2006 ; Vol. 18, No. 4. pp. 494-505.
@article{3d3ed7078d5c4fabb104d3bb324dff31,
title = "FBP: A frontier-based tree-pruning algorithm",
abstract = "A frontier-based tree-pruning algorithm (FBP) is proposed. The new method has an order of computational complexity comparable to cost-complexity pruning (CCP). Regarding tree pruning, it provides a full spectrum of information: namely, (1) given the value of the penalization parameter λ, it gives the decision tree specified by the complexity-penalization approach; (2) given the size of a decision tree, it provides the range of the penalization parameter λ, within which the complexity-penalization approach renders this tree size; (3) it finds the tree sizes that are inadmissible - no matter what the value of the penalty parameter is, the resulting tree based on a complexity-penalization framework will never have these sizes. Simulations on real data sets reveal a {"}surprise:{"} in the complexity-penalization approach, most of the tree sizes are inadmissible. FBP facilitates a more faithful implementation of cross validation (CV), which is favored by simulations. Using FBP, a stability analysis of CV is proposed.",
keywords = "Classification, Data mining, Decision trees, Tree pruning",
author = "Xiaoming Huo and Kim, {Seoung Bum} and Tsui, {Kwok Leung} and Shuchun Wang",
year = "2006",
month = "9",
day = "1",
doi = "10.1287/ijoc.1050.0133",
language = "English",
volume = "18",
pages = "494--505",
journal = "INFORMS Journal on Computing",
issn = "1091-9856",
publisher = "INFORMS Inst.for Operations Res.and the Management Sciences",
number = "4",

}

TY - JOUR

T1 - FBP

T2 - A frontier-based tree-pruning algorithm

AU - Huo, Xiaoming

AU - Kim, Seoung Bum

AU - Tsui, Kwok Leung

AU - Wang, Shuchun

PY - 2006/9/1

Y1 - 2006/9/1

N2 - A frontier-based tree-pruning algorithm (FBP) is proposed. The new method has an order of computational complexity comparable to cost-complexity pruning (CCP). Regarding tree pruning, it provides a full spectrum of information: namely, (1) given the value of the penalization parameter λ, it gives the decision tree specified by the complexity-penalization approach; (2) given the size of a decision tree, it provides the range of the penalization parameter λ, within which the complexity-penalization approach renders this tree size; (3) it finds the tree sizes that are inadmissible - no matter what the value of the penalty parameter is, the resulting tree based on a complexity-penalization framework will never have these sizes. Simulations on real data sets reveal a "surprise:" in the complexity-penalization approach, most of the tree sizes are inadmissible. FBP facilitates a more faithful implementation of cross validation (CV), which is favored by simulations. Using FBP, a stability analysis of CV is proposed.

AB - A frontier-based tree-pruning algorithm (FBP) is proposed. The new method has an order of computational complexity comparable to cost-complexity pruning (CCP). Regarding tree pruning, it provides a full spectrum of information: namely, (1) given the value of the penalization parameter λ, it gives the decision tree specified by the complexity-penalization approach; (2) given the size of a decision tree, it provides the range of the penalization parameter λ, within which the complexity-penalization approach renders this tree size; (3) it finds the tree sizes that are inadmissible - no matter what the value of the penalty parameter is, the resulting tree based on a complexity-penalization framework will never have these sizes. Simulations on real data sets reveal a "surprise:" in the complexity-penalization approach, most of the tree sizes are inadmissible. FBP facilitates a more faithful implementation of cross validation (CV), which is favored by simulations. Using FBP, a stability analysis of CV is proposed.

KW - Classification

KW - Data mining

KW - Decision trees

KW - Tree pruning

UR - http://www.scopus.com/inward/record.url?scp=33847078516&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33847078516&partnerID=8YFLogxK

U2 - 10.1287/ijoc.1050.0133

DO - 10.1287/ijoc.1050.0133

M3 - Article

AN - SCOPUS:33847078516

VL - 18

SP - 494

EP - 505

JO - INFORMS Journal on Computing

JF - INFORMS Journal on Computing

SN - 1091-9856

IS - 4

ER -